Sep 16 18:37:32.155305 ip-10-0-134-234 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 16 18:37:32.155318 ip-10-0-134-234 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 16 18:37:32.155325 ip-10-0-134-234 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 16 18:37:32.155667 ip-10-0-134-234 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 16 18:37:43.279162 ip-10-0-134-234 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 16 18:37:43.279181 ip-10-0-134-234 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot eda4ec31443d4bdd95fae5682055d36c --
Sep 16 18:39:32.100544 ip-10-0-134-234 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 18:39:32.509305 ip-10-0-134-234 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:39:32.509305 ip-10-0-134-234 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 18:39:32.509305 ip-10-0-134-234 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:39:32.509305 ip-10-0-134-234 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 18:39:32.509305 ip-10-0-134-234 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:39:32.510370 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.510282    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 18:39:32.513086 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513069    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:39:32.513121 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513088    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:39:32.513121 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513093    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:39:32.513121 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513096    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:39:32.513121 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513100    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:39:32.513121 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513103    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:39:32.513121 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513106    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:39:32.513121 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513109    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:39:32.513121 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513112    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:39:32.513121 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513115    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:39:32.513121 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513118    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:39:32.513121 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513121    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:39:32.513121 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513124    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:39:32.513121 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513127    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513130    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513133    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513136    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513138    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513141    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513148    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513151    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513153    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513156    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513158    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513161    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513164    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513166    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513183    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513187    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513190    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513195    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513199    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513203    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:39:32.513450 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513207    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513209    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513212    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513215    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513217    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513220    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513222    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513225    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513229    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513231    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513234    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513236    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513239    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513242    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513245    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513255    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513258    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513260    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513263    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513266    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:39:32.513978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513269    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513273    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513275    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513278    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513281    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513284    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513287    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513289    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513292    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513294    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513297    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513301    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513305    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513308    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513311    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513313    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513316    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513319    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513322    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:39:32.514721 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513324    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513327    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513330    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513333    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513335    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513338    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513341    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513346    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513351    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513354    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513357    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513360    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513363    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513366    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513369    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513372    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513375    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513377    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513380    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513383    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:39:32.515289 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513386    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513388    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513391    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513394    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513397    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513400    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.513402    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515569    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515581    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515584    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515588    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515591    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515594    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515597    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515600    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515603    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515607    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515612    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515615    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:39:32.515772 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515618    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515621    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515624    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515626    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515629    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515632    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515634    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515637    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515639    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515642    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515644    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515647    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515650    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515652    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515655    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515658    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515660    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515662    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515666    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515669    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:39:32.516286 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515672    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515675    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515678    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515680    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515683    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515686    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515688    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515691    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515694    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515697    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515700    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515702    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515705    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515708    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515710    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515713    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515715    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515718    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515720    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515723    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:39:32.516780 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515725    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515728    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515731    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515734    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515736    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515739    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515741    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515744    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515746    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515749    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515751    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515755    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515757    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515761    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515764    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515766    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515769    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515771    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515774    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515777    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515779    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:39:32.517319 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515782    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515785    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515788    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515790    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515793    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515795    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515798    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515802    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515805    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515807    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515811    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515813    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515816    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515820    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515823    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515825    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515827    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515830    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:39:32.517905 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515834    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515838    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515841    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515844    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515847    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515850    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515853    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.515856    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516292    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516299    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516303    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516306    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516309    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516312    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516315    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516318    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516321    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516324    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516326    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:39:32.518364 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516329    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516332    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516334    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516338    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516342    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516345    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516349    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516353    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516356    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516358    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516361    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516364    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516366    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516369    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516372    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516375    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516377    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516380    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516383    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:39:32.518824 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516386    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516389    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516392    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516394    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516397    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516400    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516402    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516405    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516407    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516411    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516414    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516416    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516419    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516421    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516424    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516427    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516429    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516432    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516434    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516437    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:39:32.519293 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516440    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516443    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516445    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516448    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516451    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516453    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516457    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516460    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516463    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516466    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516468    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516471    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516473    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516476    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516479    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516481    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516484    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516487    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516489    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516492    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:39:32.519769 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516494    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516497    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516499    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516502    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516504    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516507    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516510    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516512    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516514    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516517    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516519    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516522    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516526    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516529    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516532    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516535    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516538    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516541    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516544    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:39:32.520265 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516547    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516550    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516553    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516556    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516558    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516561    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516564    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516567    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516570    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.516572    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516640    2578 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516679    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516685    2578 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516690    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516694    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516698    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516707    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516711    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516714    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516718    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516721    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 18:39:32.520724 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516724    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516727    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516730    2578 flags.go:64] FLAG: --cgroup-root=""
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516733    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516736    2578 flags.go:64] FLAG: --client-ca-file=""
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516740    2578 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516743    2578 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516748    2578 flags.go:64] FLAG: --cluster-domain=""
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516752    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516756    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516759    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516763    2578 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516767    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516770    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516773    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516777    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516780    2578 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516783    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516786    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516790    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516793    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516797    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516800    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516803    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516806    2578 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 18:39:32.521294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516809    2578 flags.go:64] FLAG: --enable-server="true"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516812    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516816    2578 flags.go:64] FLAG: --event-burst="100"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516819    2578 flags.go:64] FLAG: --event-qps="50"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516822    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516825    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516828    2578 flags.go:64] FLAG: --eviction-hard=""
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516832    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516835    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516838    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516841    2578 flags.go:64] FLAG: --eviction-soft=""
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516844    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516847    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516850    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516853    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516856    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516859    2578 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516865    2578 flags.go:64] FLAG: --feature-gates=""
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516868    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516871    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516874    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516878    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516881    2578 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516884    2578 flags.go:64] FLAG: --help="false"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516887    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-134-234.ec2.internal"
Sep 16 18:39:32.521893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516890    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516893    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516896    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516899    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516902    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516905    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516908    2578 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516911    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516913    2578 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516916    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516919    2578 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516922    2578 flags.go:64] FLAG: --kube-reserved=""
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516925    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516928    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516931    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516934    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516937    2578 flags.go:64] FLAG: --lock-file=""
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516939    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516942    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516945    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516951    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516954    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516957    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 18:39:32.522530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516960    2578 flags.go:64] FLAG: --logging-format="text"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516965    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516968    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516971    2578 flags.go:64] FLAG: --manifest-url=""
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516974    2578 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516978    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516981    2578 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516985    2578 flags.go:64] FLAG: --max-pods="110"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516988    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516991    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516994    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516996    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.516999    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517003    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517006    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517013    2578 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517016    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517019    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517022    2578 flags.go:64] FLAG: --pod-cidr=""
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517025    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517031    2578 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517034    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517037    2578 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517040    2578 flags.go:64] FLAG: --port="10250"
Sep 16 18:39:32.523101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517043    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517046    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0def4e7a6e4ae35b0"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517049    2578 flags.go:64] FLAG: --qos-reserved=""
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517052    2578 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517055    2578 flags.go:64] FLAG: --register-node="true"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517058    2578 flags.go:64] FLAG: --register-with-taints=""
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517061    2578 flags.go:64] FLAG: --registry-burst="10"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517066    2578 flags.go:64] FLAG: --registry-qps="5"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517069    2578 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517073    2578 flags.go:64] FLAG: --reserved-memory=""
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517076    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517080    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517083    2578 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517086    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517089    2578 flags.go:64] FLAG: --runonce="false"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517092    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517094    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517097    2578 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517100    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517103    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517106    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517109    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517112    2578 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517115    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517118    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 18:39:32.523723 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517121    2578 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517123    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517127    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517130    2578 flags.go:64] FLAG: --system-cgroups=""
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517133    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517138    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517141    2578 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517144    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517150    2578 flags.go:64] FLAG: --tls-min-version=""
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517153    2578 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517156    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517159    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517162    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517165    2578 flags.go:64] FLAG: --v="2"
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517186    2578 flags.go:64] FLAG: --version="false"
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517191    2578 flags.go:64] FLAG: --vmodule=""
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517196    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517199    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517345    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517350    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517353    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517356    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517360    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:39:32.524793 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517363    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517366    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517370    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517372    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517375    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517378    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517381    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517384    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517386    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517389    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517394    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517396    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517399    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517402    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517404    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517407    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517409    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517412    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517415    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517418    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:39:32.525869 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517421    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517424    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517426    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517429    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517432    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517437    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517441    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517446    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517449    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517452    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517455    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517458    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517460    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517464    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517467    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517470    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517473    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517476    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517479    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517482    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:39:32.526746 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517484    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517487    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517491    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517494    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517496    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517499    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517502    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517504    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517507    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517509    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517512    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517514    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517517    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517520    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517522    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517525    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517528    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517530    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517534    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517537    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:39:32.527654 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517540    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517543    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517545    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517548    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517550    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517553    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517556    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517559    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517561    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517563    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517566    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517570    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517573    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517576    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517580    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517583    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517585    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517588    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517590    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:39:32.528320 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517593    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:39:32.529105 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517596    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:39:32.529105 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517598    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:39:32.529105 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517601    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:39:32.529105 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517604    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:39:32.529105 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517607    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:39:32.529105 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517609    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:39:32.529105 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517612    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:39:32.529105 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517615    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:39:32.529105 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517618    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:39:32.529105 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517620    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:39:32.529105 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517623    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:39:32.529105 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517626    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:39:32.529105 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517629    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:39:32.529105 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.517632    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:39:32.529105 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.517637    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.526397    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.526416    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.526444    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.526453    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526535    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526543    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526548    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526553    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526558    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526563    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526567    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526572    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526576    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526581    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526585    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526589    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526593    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526598    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526604    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:39:32.529771 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526608    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526612    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526616    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526621    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526626    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526632    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526638    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526644    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526651    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526656    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526662    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526667    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526672    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526677    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526684    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526688    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526693    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526697    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526701    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:39:32.530369 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526705    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526710    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526714    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526718    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526723    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526727    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526731    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526735    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526739    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526743    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526747    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526751    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526755    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526759    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526762    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526766    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526770    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526774    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526779    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526783    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:39:32.530942 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526787    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526791    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526795    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526799    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526804    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526808    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526813    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526818    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526823    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526828    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526833    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526837    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526841    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526846    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526850    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526854    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526858    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526863    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526867    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526871    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:39:32.531576 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526876    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526880    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526884    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526888    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526892    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526897    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526901    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526905    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526910    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526915    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526919    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526923    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526927    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526932    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526938    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526943    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526948    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526952    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526958    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:39:32.532065 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526963    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:39:32.532616 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526968    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:39:32.532616 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526974    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:39:32.532616 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526978    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:39:32.532616 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526983    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:39:32.532616 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.526987    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:39:32.532616 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.526996    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:39:32.532616 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527190    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:39:32.532616 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527200    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:39:32.532616 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527205    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:39:32.532616 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527209    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:39:32.532616 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527213    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:39:32.532616 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527217    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:39:32.532616 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527222    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:39:32.532616 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527226    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:39:32.532616 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527229    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527234    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527239    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527244    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527248    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527253    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527257    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527263    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527269    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527275    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527280    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527284    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527288    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527293    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527297    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527301    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527308    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527314    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527319    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527324    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:39:32.532975 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527330    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527335    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527339    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527343    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527347    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527353    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527359    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527364    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527369    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527373    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527378    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527382    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527387    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527391    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527396    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527400    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527405    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527409    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527414    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:39:32.533512 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527418    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527422    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527426    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527430    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527435    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527439    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527443    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527447    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527451    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527456    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527460    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527466    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527470    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527474    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527479    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527483    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527487    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527491    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527496    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527500    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:39:32.534036 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527505    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527509    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527513    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527518    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527522    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527526    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527530    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527534    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527539    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527544    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527548    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527552    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527556    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527560    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527564    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527568    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527572    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527576    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527580    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527585    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:39:32.534866 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527589    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:39:32.535396 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527593    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:39:32.535396 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527598    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:39:32.535396 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527602    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:39:32.535396 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527607    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:39:32.535396 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527611    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:39:32.535396 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527615    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:39:32.535396 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527620    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:39:32.535396 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527625    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:39:32.535396 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527629    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:39:32.535396 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527633    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:39:32.535396 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:32.527637    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:39:32.535396 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.527646    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:39:32.535396 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.527807    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 18:39:32.535396 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.530315    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 18:39:32.535396 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.531263    2578 server.go:1019] "Starting client certificate rotation"
Sep 16 18:39:32.535763 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.531367    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 18:39:32.535763 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.532326    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 18:39:32.535763 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.534588    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 18:39:32.537876 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.537853    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 18:39:32.553918 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.553897    2578 log.go:25] "Validated CRI v1 runtime API"
Sep 16 18:39:32.556602 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.556587    2578 log.go:25] "Validated CRI v1 image API"
Sep 16 18:39:32.557458 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.557441    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 18:39:32.560588 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.560567    2578 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 aa747753-bcc8-4e6b-860b-447c384ab118:/dev/nvme0n1p3 ce2c353f-2ae4-48a8-8e3c-574c32951f09:/dev/nvme0n1p4]
Sep 16 18:39:32.560676 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.560587    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 18:39:32.565732 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.565607    2578 manager.go:216] Machine: {Timestamp:2026-09-16 18:39:32.564266389 +0000 UTC m=+0.351830546 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3109872 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a0babf55a81bc805a79dabf804197 SystemUUID:ec2a0bab-f55a-81bc-805a-79dabf804197 BootID:eda4ec31-443d-4bdd-95fa-e5682055d36c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c9:c1:c2:75 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c9:c1:c2:75 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:b1:e6:be:2c:eb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 18:39:32.565732 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.565724    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 18:39:32.565894 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.565841    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 18:39:32.566851 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.566832    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 18:39:32.568773 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.568743    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 18:39:32.568918 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.568776    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-234.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 18:39:32.569506 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.569495    2578 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 18:39:32.569545 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.569509    2578 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 18:39:32.569545 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.569516    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 18:39:32.570141 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.570131    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 18:39:32.570861 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.570851    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 18:39:32.574589 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.574577    2578 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 18:39:32.574700 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.574690    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 18:39:32.574732 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.574721    2578 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 18:39:32.574762 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.574736    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 18:39:32.574762 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.574751    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 18:39:32.574762 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.574761    2578 kubelet.go:398] "Adding apiserver pod source"
Sep 16 18:39:32.574852 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.574769    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 18:39:32.575117 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.575105    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 18:39:32.575165 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.575124    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 18:39:32.577679 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.577663    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 18:39:32.579546 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.579530    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 18:39:32.579592 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.579562    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 18:39:32.581749 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.581735    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 18:39:32.581793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.581759    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 18:39:32.581793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.581766    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 18:39:32.581793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.581772    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 18:39:32.581793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.581780    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 18:39:32.581793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.581786    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 18:39:32.581931 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.581800    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 18:39:32.581931 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.581805    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 18:39:32.581931 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.581813    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 18:39:32.581931 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.581819    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 18:39:32.581931 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.581830    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 18:39:32.581931 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.581835    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 18:39:32.582759 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.582743    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 18:39:32.582759 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.582753    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 18:39:32.583552 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.583528    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 18:39:32.583841 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.583821    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-234.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 18:39:32.586455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.586442    2578 server.go:1268] "Started kubelet"
Sep 16 18:39:32.586581 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.586515    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 18:39:32.586633 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.586570    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 18:39:32.590696 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.590670    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 18:39:32.591246 ip-10-0-134-234 systemd[1]: Started Kubernetes Kubelet.
Sep 16 18:39:32.592626 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.592609    2578 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 18:39:32.593161 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.593140    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 18:39:32.596215 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.596195    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m4kms"
Sep 16 18:39:32.597996 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.597967    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 18:39:32.599504 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.599483    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-234.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 18:39:32.600010 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.599991    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 18:39:32.600925 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.600913    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 18:39:32.600986 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.599533    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-234.ec2.internal.18d5e12435bee135  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-234.ec2.internal,UID:ip-10-0-134-234.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-234.ec2.internal,},FirstTimestamp:2026-09-16 18:39:32.586414389 +0000 UTC m=+0.373978535,LastTimestamp:2026-09-16 18:39:32.586414389 +0000 UTC m=+0.373978535,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-234.ec2.internal,}"
Sep 16 18:39:32.601078 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.601032    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-234.ec2.internal\" not found"
Sep 16 18:39:32.601631 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.601611    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 18:39:32.601631 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.601614    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 18:39:32.601779 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.601640    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 18:39:32.601779 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.601666    2578 factory.go:55] Registering systemd factory
Sep 16 18:39:32.601779 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.601679    2578 factory.go:223] Registration of the systemd container factory successfully
Sep 16 18:39:32.601779 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.601718    2578 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 18:39:32.601779 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.601731    2578 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 18:39:32.601982 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.601910    2578 factory.go:153] Registering CRI-O factory
Sep 16 18:39:32.601982 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.601918    2578 factory.go:223] Registration of the crio container factory successfully
Sep 16 18:39:32.601982 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.601970    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 18:39:32.601982 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.601984    2578 factory.go:103] Registering Raw factory
Sep 16 18:39:32.602184 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.601997    2578 manager.go:1196] Started watching for new ooms in manager
Sep 16 18:39:32.602405 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.602391    2578 manager.go:349] Starting recovery of all containers
Sep 16 18:39:32.604369 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.604348    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m4kms"
Sep 16 18:39:32.607957 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.607917    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 18:39:32.608982 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.608471    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-234.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 18:39:32.608982 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.608658    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 16 18:39:32.613556 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.613377    2578 manager.go:354] Recovery completed
Sep 16 18:39:32.619771 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.619714    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:39:32.621850 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.621826    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:39:32.621937 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.621866    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:39:32.621937 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.621880    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:39:32.622594 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.622580    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:39:32.622686 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.622578    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 18:39:32.622686 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.622608    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 18:39:32.622686 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.622623    2578 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 18:39:32.623441 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.623423    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:39:32.623529 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.623454    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:39:32.623529 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.623467    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:39:32.623782 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.623769    2578 policy_none.go:49] "None policy: Start"
Sep 16 18:39:32.623814 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.623785    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 18:39:32.623814 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.623795    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 18:39:32.625217 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.625200    2578 policy_none.go:47] "Start"
Sep 16 18:39:32.647770 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.647752    2578 manager.go:337] "Starting Device Plugin manager"
Sep 16 18:39:32.647876 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.647798    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 18:39:32.647876 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.647810    2578 server.go:87] "Starting device plugin registration server"
Sep 16 18:39:32.648154 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.648141    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 18:39:32.648213 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.648158    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 18:39:32.648690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.648340    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 18:39:32.648690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.648414    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 18:39:32.648690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.648422    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 18:39:32.649343 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.649323    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 18:39:32.649436 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.649376    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-234.ec2.internal\" not found"
Sep 16 18:39:32.702225 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.702191    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 18:39:32.702400 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.702251    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 18:39:32.702400 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.702277    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 18:39:32.702400 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.702326    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 18:39:32.704513 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.704489    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:39:32.748301 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.748264    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:39:32.749386 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.749369    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:39:32.749478 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.749401    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:39:32.749478 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.749413    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:39:32.749478 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.749435    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-234.ec2.internal"
Sep 16 18:39:32.755701 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.755682    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-234.ec2.internal"
Sep 16 18:39:32.755774 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.755711    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-234.ec2.internal\": node \"ip-10-0-134-234.ec2.internal\" not found"
Sep 16 18:39:32.768940 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.768866    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-234.ec2.internal\" not found"
Sep 16 18:39:32.803361 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.803316    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-234.ec2.internal"]
Sep 16 18:39:32.803494 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.803413    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:39:32.804880 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.804863    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:39:32.804980 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.804895    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:39:32.804980 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.804910    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:39:32.806080 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.806065    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:39:32.806201 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.806185    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal"
Sep 16 18:39:32.806260 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.806214    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:39:32.806843 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.806826    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:39:32.806925 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.806860    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:39:32.806925 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.806872    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:39:32.806925 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.806826    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:39:32.807017 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.806934    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:39:32.807017 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.806950    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:39:32.808203 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.808190    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-234.ec2.internal"
Sep 16 18:39:32.808252 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.808215    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:39:32.808991 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.808968    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:39:32.808991 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.808987    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:39:32.809090 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.808999    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:39:32.831009 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.830989    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-234.ec2.internal\" not found" node="ip-10-0-134-234.ec2.internal"
Sep 16 18:39:32.834360 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.834343    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-234.ec2.internal\" not found" node="ip-10-0-134-234.ec2.internal"
Sep 16 18:39:32.869413 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.869375    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-234.ec2.internal\" not found"
Sep 16 18:39:32.902643 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.902609    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/41ec96924e9815932b677eedb2f521c7-config\") pod \"kube-apiserver-proxy-ip-10-0-134-234.ec2.internal\" (UID: \"41ec96924e9815932b677eedb2f521c7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-234.ec2.internal"
Sep 16 18:39:32.902643 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.902639    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b8171f0e1a50a19cc3561ab6c613a155-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal\" (UID: \"b8171f0e1a50a19cc3561ab6c613a155\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal"
Sep 16 18:39:32.902884 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:32.902661    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8171f0e1a50a19cc3561ab6c613a155-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal\" (UID: \"b8171f0e1a50a19cc3561ab6c613a155\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal"
Sep 16 18:39:32.969694 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:32.969666    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-234.ec2.internal\" not found"
Sep 16 18:39:33.003720 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.003687    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b8171f0e1a50a19cc3561ab6c613a155-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal\" (UID: \"b8171f0e1a50a19cc3561ab6c613a155\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal"
Sep 16 18:39:33.003799 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.003721    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8171f0e1a50a19cc3561ab6c613a155-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal\" (UID: \"b8171f0e1a50a19cc3561ab6c613a155\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal"
Sep 16 18:39:33.003843 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.003789    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b8171f0e1a50a19cc3561ab6c613a155-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal\" (UID: \"b8171f0e1a50a19cc3561ab6c613a155\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal"
Sep 16 18:39:33.003843 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.003802    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8171f0e1a50a19cc3561ab6c613a155-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal\" (UID: \"b8171f0e1a50a19cc3561ab6c613a155\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal"
Sep 16 18:39:33.003843 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.003833    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/41ec96924e9815932b677eedb2f521c7-config\") pod \"kube-apiserver-proxy-ip-10-0-134-234.ec2.internal\" (UID: \"41ec96924e9815932b677eedb2f521c7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-234.ec2.internal"
Sep 16 18:39:33.003952 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.003861    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/41ec96924e9815932b677eedb2f521c7-config\") pod \"kube-apiserver-proxy-ip-10-0-134-234.ec2.internal\" (UID: \"41ec96924e9815932b677eedb2f521c7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-234.ec2.internal"
Sep 16 18:39:33.070225 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:33.070116    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-234.ec2.internal\" not found"
Sep 16 18:39:33.133776 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.133743    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal"
Sep 16 18:39:33.136858 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.136829    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-234.ec2.internal"
Sep 16 18:39:33.171166 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:33.171133    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-234.ec2.internal\" not found"
Sep 16 18:39:33.276261 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:33.276220    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-234.ec2.internal\" not found"
Sep 16 18:39:33.376482 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:33.376453    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-234.ec2.internal\" not found"
Sep 16 18:39:33.388858 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.388837    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:39:33.402220 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.402199    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-234.ec2.internal"
Sep 16 18:39:33.415600 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.415578    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 18:39:33.417370 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.417353    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal"
Sep 16 18:39:33.434283 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.434263    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 18:39:33.530669 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.530632    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 18:39:33.531188 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.530822    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 18:39:33.531188 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.530834    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 18:39:33.558656 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.558629    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:39:33.575428 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.575405    2578 apiserver.go:52] "Watching apiserver"
Sep 16 18:39:33.583640 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.583616    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 18:39:33.584153 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.584135    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:39:33.585668 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.585644    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-sdmdp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6","openshift-dns/node-resolver-57l9p","openshift-image-registry/node-ca-478xv","openshift-multus/multus-qx9nj","openshift-multus/network-metrics-daemon-282m9","openshift-network-diagnostics/network-check-target-wbfzv","openshift-ovn-kubernetes/ovnkube-node-gbxkd","kube-system/global-pull-secret-syncer-c8l7h","kube-system/kube-apiserver-proxy-ip-10-0-134-234.ec2.internal","openshift-cluster-node-tuning-operator/tuned-v8fzr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal","openshift-multus/multus-additional-cni-plugins-nsrzq","openshift-network-operator/iptables-alerter-h5qsz"]
Sep 16 18:39:33.587382 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.587363    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sdmdp"
Sep 16 18:39:33.588674 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.588647    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.589818 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.589796    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-57l9p"
Sep 16 18:39:33.590994 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.590975    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-478xv"
Sep 16 18:39:33.594095 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.594021    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.594294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.594214    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bqw9q\""
Sep 16 18:39:33.595445 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.595424    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:33.595445 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.595439    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 18:39:33.595597 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:33.595539    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:39:33.596126 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.596104    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 18:39:33.596225 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.596145    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 18:39:33.596225 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.596168    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 18:39:33.596225 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.596213    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 18:39:33.596396 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.596145    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 18:39:33.596396 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.596111    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mv4z6\""
Sep 16 18:39:33.596396 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.596215    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r6cmg\""
Sep 16 18:39:33.596579 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.596495    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 18:39:33.596579 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.596498    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 18:39:33.596579 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.596498    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 18:39:33.596579 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.596536    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 18:39:33.597065 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.596853    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:33.597065 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:33.596926    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:39:33.597065 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.597024    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 18:39:33.597065 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.597037    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 18:39:33.597315 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.597170    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 18:39:33.597315 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.597299    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mm4l2\""
Sep 16 18:39:33.597380 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.597335    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 18:39:33.598476 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.598393    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zzzlq\""
Sep 16 18:39:33.598998 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.598915    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.599945 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.599926    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:33.600040 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:33.599984    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:39:33.600157 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.600119    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 18:39:33.601331 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.601314    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.601599 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.601581    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 18:39:33.601721 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.601702    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 18:39:33.602346 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.602328    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dqvhx\""
Sep 16 18:39:33.602479 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.602402    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 18:39:33.602479 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.602407    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 18:39:33.602479 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.602413    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 18:39:33.603516 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.603246    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.603516 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.603298    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:39:33.603516 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.603403    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 18:39:33.603765 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.603750    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-v8lms\""
Sep 16 18:39:33.604220 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.604189    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 18:39:33.604678 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.604660    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h5qsz"
Sep 16 18:39:33.606831 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.606814    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 18:39:33.607002 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.606990    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:39:33.607182 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.607165    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 18:39:33.607527 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.607497    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 18:39:33.607733 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.607715    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kfczg\""
Sep 16 18:39:33.607800 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.607713    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-var-lib-cni-bin\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.607800 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.607769    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-etc-kubernetes\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.607800 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.607795    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-run-ovn-kubernetes\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.607901 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.607848    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret\") pod \"global-pull-secret-syncer-c8l7h\" (UID: \"73a1706e-dce9-475e-b553-642999cc42c0\") " pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:33.607949 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.607903    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpgjd\" (UniqueName: \"kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd\") pod \"network-check-target-wbfzv\" (UID: \"2aad88e5-b35d-4876-8a3b-c8b0b64f8570\") " pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:33.607949 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.607921    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-sysconfig\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.607949 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.607937    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-kubernetes\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.608067 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.607948    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 18:39:33.608067 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.607952    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-systemd-units\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.608067 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.607988    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zkfv\" (UniqueName: \"kubernetes.io/projected/87684818-77c6-45bc-bee8-61a206348549-kube-api-access-6zkfv\") pod \"network-metrics-daemon-282m9\" (UID: \"87684818-77c6-45bc-bee8-61a206348549\") " pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:33.608067 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608010    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-modprobe-d\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.608067 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608026    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-run-multus-certs\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.608067 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608049    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-slash\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.608362 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608072    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-node-log\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.608362 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608095    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-cni-netd\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.608362 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608116    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1df53334-7b61-47d0-9787-1dc604c84896-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.608362 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608135    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-registration-dir\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.608362 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608204    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5bfaaaff-fed0-456e-a757-8655bc43f26d-ovn-node-metrics-cert\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.608362 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608237    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvqd7\" (UniqueName: \"kubernetes.io/projected/5bfaaaff-fed0-456e-a757-8655bc43f26d-kube-api-access-tvqd7\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.608362 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608266    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/73a1706e-dce9-475e-b553-642999cc42c0-kubelet-config\") pod \"global-pull-secret-syncer-c8l7h\" (UID: \"73a1706e-dce9-475e-b553-642999cc42c0\") " pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:33.608362 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608293    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ffef2c6f-fe5b-4b5e-9d95-e7eac9b4f8be-agent-certs\") pod \"konnectivity-agent-sdmdp\" (UID: \"ffef2c6f-fe5b-4b5e-9d95-e7eac9b4f8be\") " pod="kube-system/konnectivity-agent-sdmdp"
Sep 16 18:39:33.608362 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608318    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-etc-openvswitch\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.608362 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608347    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-multus-cni-dir\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.608783 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608402    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-run-netns\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.608783 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608397    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 18:34:33 +0000 UTC" deadline="2028-05-06 23:23:27.102338749 +0000 UTC"
Sep 16 18:39:33.608783 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608437    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-var-lib-openvswitch\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.608783 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608442    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14356h43m53.493900762s"
Sep 16 18:39:33.608783 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608481    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcc9v\" (UniqueName: \"kubernetes.io/projected/366fd53e-0cd4-4fc9-8737-00383ad97707-kube-api-access-kcc9v\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.608783 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608519    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1df53334-7b61-47d0-9787-1dc604c84896-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.608783 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608546    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.608783 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608573    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-run-k8s-cni-cncf-io\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.608783 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608607    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-multus-conf-dir\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.608783 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608662    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5da3f960-f4f8-4301-89ef-1823ec93bff5-multus-daemon-config\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.608783 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608721    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-run-netns\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.608783 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608753    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-run-openvswitch\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.608783 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608783    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-sysctl-d\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608811    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-etc-selinux\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608840    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-cni-bin\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608869    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-tuned\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608902    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1df53334-7b61-47d0-9787-1dc604c84896-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608921    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6792\" (UniqueName: \"kubernetes.io/projected/71dda4a6-6301-4bf9-abb7-b318f57fe02e-kube-api-access-d6792\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608938    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fae250ed-5b59-4388-af13-fcc8d0446e5d-host\") pod \"node-ca-478xv\" (UID: \"fae250ed-5b59-4388-af13-fcc8d0446e5d\") " pod="openshift-image-registry/node-ca-478xv"
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608955    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-468sz\" (UniqueName: \"kubernetes.io/projected/fae250ed-5b59-4388-af13-fcc8d0446e5d-kube-api-access-468sz\") pod \"node-ca-478xv\" (UID: \"fae250ed-5b59-4388-af13-fcc8d0446e5d\") " pod="openshift-image-registry/node-ca-478xv"
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.608995    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-hostroot\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609019    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-run\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609035    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-lib-modules\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609051    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1df53334-7b61-47d0-9787-1dc604c84896-system-cni-dir\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609074    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-sys-fs\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609079    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zgd4x\""
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609113    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-system-cni-dir\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609133    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d96fa3e0-be40-4f37-99be-57261dce04d1-tmp-dir\") pod \"node-resolver-57l9p\" (UID: \"d96fa3e0-be40-4f37-99be-57261dce04d1\") " pod="openshift-dns/node-resolver-57l9p"
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609156    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-log-socket\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.609198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609182    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/73a1706e-dce9-475e-b553-642999cc42c0-dbus\") pod \"global-pull-secret-syncer-c8l7h\" (UID: \"73a1706e-dce9-475e-b553-642999cc42c0\") " pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:33.609792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609221    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-sysctl-conf\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.609792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609241    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ffef2c6f-fe5b-4b5e-9d95-e7eac9b4f8be-konnectivity-ca\") pod \"konnectivity-agent-sdmdp\" (UID: \"ffef2c6f-fe5b-4b5e-9d95-e7eac9b4f8be\") " pod="kube-system/konnectivity-agent-sdmdp"
Sep 16 18:39:33.609792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609263    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-multus-socket-dir-parent\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.609792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609300    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.609792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609320    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5bfaaaff-fed0-456e-a757-8655bc43f26d-ovnkube-config\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.609792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609361    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs\") pod \"network-metrics-daemon-282m9\" (UID: \"87684818-77c6-45bc-bee8-61a206348549\") " pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:33.609792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609383    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-socket-dir\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.609792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609409    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-var-lib-kubelet\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.609792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609433    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1df53334-7b61-47d0-9787-1dc604c84896-cnibin\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.609792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609464    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1df53334-7b61-47d0-9787-1dc604c84896-cni-binary-copy\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.609792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609494    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-run-ovn\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.609792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609521    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wthl\" (UniqueName: \"kubernetes.io/projected/5da3f960-f4f8-4301-89ef-1823ec93bff5-kube-api-access-5wthl\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.609792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609552    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv77m\" (UniqueName: \"kubernetes.io/projected/1df53334-7b61-47d0-9787-1dc604c84896-kube-api-access-lv77m\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.609792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609567    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d96fa3e0-be40-4f37-99be-57261dce04d1-hosts-file\") pod \"node-resolver-57l9p\" (UID: \"d96fa3e0-be40-4f37-99be-57261dce04d1\") " pod="openshift-dns/node-resolver-57l9p"
Sep 16 18:39:33.609792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609589    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-cnibin\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.609792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609634    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-var-lib-kubelet\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.610351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609652    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5bfaaaff-fed0-456e-a757-8655bc43f26d-ovnkube-script-lib\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.610351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609681    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/366fd53e-0cd4-4fc9-8737-00383ad97707-tmp\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.610351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-device-dir\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.610351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609718    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckj94\" (UniqueName: \"kubernetes.io/projected/d96fa3e0-be40-4f37-99be-57261dce04d1-kube-api-access-ckj94\") pod \"node-resolver-57l9p\" (UID: \"d96fa3e0-be40-4f37-99be-57261dce04d1\") " pod="openshift-dns/node-resolver-57l9p"
Sep 16 18:39:33.610351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609752    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fae250ed-5b59-4388-af13-fcc8d0446e5d-serviceca\") pod \"node-ca-478xv\" (UID: \"fae250ed-5b59-4388-af13-fcc8d0446e5d\") " pod="openshift-image-registry/node-ca-478xv"
Sep 16 18:39:33.610351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609775    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-var-lib-cni-multus\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.610351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609790    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-kubelet\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.610351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609805    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-run-systemd\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.610351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609820    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-sys\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.610351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609842    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-host\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.610351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609868    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5da3f960-f4f8-4301-89ef-1823ec93bff5-cni-binary-copy\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.610351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609894    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5bfaaaff-fed0-456e-a757-8655bc43f26d-env-overrides\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.610351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.609909    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-systemd\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.620245 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.620225    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 18:39:33.641449 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.641425    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-js2qk"
Sep 16 18:39:33.650685 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.650664    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-js2qk"
Sep 16 18:39:33.685109 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:33.685078    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ec96924e9815932b677eedb2f521c7.slice/crio-07063be31da41a974b638783403560bef83f463643b9b25e6437a415ba5e4708 WatchSource:0}: Error finding container 07063be31da41a974b638783403560bef83f463643b9b25e6437a415ba5e4708: Status 404 returned error can't find the container with id 07063be31da41a974b638783403560bef83f463643b9b25e6437a415ba5e4708
Sep 16 18:39:33.688013 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:33.687933    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8171f0e1a50a19cc3561ab6c613a155.slice/crio-e8ec14cec9a06846dd6c10e4a7382e1ba501c97eba6d9b393264015315de1c08 WatchSource:0}: Error finding container e8ec14cec9a06846dd6c10e4a7382e1ba501c97eba6d9b393264015315de1c08: Status 404 returned error can't find the container with id e8ec14cec9a06846dd6c10e4a7382e1ba501c97eba6d9b393264015315de1c08
Sep 16 18:39:33.694064 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.694040    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:39:33.702337 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.702318    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 18:39:33.705743 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.705696    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-234.ec2.internal" event={"ID":"41ec96924e9815932b677eedb2f521c7","Type":"ContainerStarted","Data":"07063be31da41a974b638783403560bef83f463643b9b25e6437a415ba5e4708"}
Sep 16 18:39:33.707049 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.707025    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal" event={"ID":"b8171f0e1a50a19cc3561ab6c613a155","Type":"ContainerStarted","Data":"e8ec14cec9a06846dd6c10e4a7382e1ba501c97eba6d9b393264015315de1c08"}
Sep 16 18:39:33.710062 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710040    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-etc-openvswitch\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.710211 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710079    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-multus-cni-dir\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.710211 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710103    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-run-netns\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.710211 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710127    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-var-lib-openvswitch\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.710211 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcc9v\" (UniqueName: \"kubernetes.io/projected/366fd53e-0cd4-4fc9-8737-00383ad97707-kube-api-access-kcc9v\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.710211 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710159    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-etc-openvswitch\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.710211 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710203    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-multus-cni-dir\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710214    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-var-lib-openvswitch\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710241    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-run-netns\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710284    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1df53334-7b61-47d0-9787-1dc604c84896-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710319    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-run-k8s-cni-cncf-io\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710368    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-multus-conf-dir\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710391    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5da3f960-f4f8-4301-89ef-1823ec93bff5-multus-daemon-config\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710384    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710416    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-run-netns\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710419    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-multus-conf-dir\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710423    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-run-k8s-cni-cncf-io\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710440    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-run-openvswitch\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710464    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-sysctl-d\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710476    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-run-netns\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710489    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-etc-selinux\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710513    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-cni-bin\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.710553 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710518    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-run-openvswitch\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710534    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-tuned\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710572    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-cni-bin\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710583    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-etc-selinux\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710602    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-sysctl-d\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1df53334-7b61-47d0-9787-1dc604c84896-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710638    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6792\" (UniqueName: \"kubernetes.io/projected/71dda4a6-6301-4bf9-abb7-b318f57fe02e-kube-api-access-d6792\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710665    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fae250ed-5b59-4388-af13-fcc8d0446e5d-host\") pod \"node-ca-478xv\" (UID: \"fae250ed-5b59-4388-af13-fcc8d0446e5d\") " pod="openshift-image-registry/node-ca-478xv"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710690    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-468sz\" (UniqueName: \"kubernetes.io/projected/fae250ed-5b59-4388-af13-fcc8d0446e5d-kube-api-access-468sz\") pod \"node-ca-478xv\" (UID: \"fae250ed-5b59-4388-af13-fcc8d0446e5d\") " pod="openshift-image-registry/node-ca-478xv"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710715    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-hostroot\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710737    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-run\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710762    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-lib-modules\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1df53334-7b61-47d0-9787-1dc604c84896-system-cni-dir\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710814    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-sys-fs\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710800    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1df53334-7b61-47d0-9787-1dc604c84896-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710840    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-system-cni-dir\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710865    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d96fa3e0-be40-4f37-99be-57261dce04d1-tmp-dir\") pod \"node-resolver-57l9p\" (UID: \"d96fa3e0-be40-4f37-99be-57261dce04d1\") " pod="openshift-dns/node-resolver-57l9p"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710878    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-run\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.711554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710889    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-log-socket\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710914    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/73a1706e-dce9-475e-b553-642999cc42c0-dbus\") pod \"global-pull-secret-syncer-c8l7h\" (UID: \"73a1706e-dce9-475e-b553-642999cc42c0\") " pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710936    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fae250ed-5b59-4388-af13-fcc8d0446e5d-host\") pod \"node-ca-478xv\" (UID: \"fae250ed-5b59-4388-af13-fcc8d0446e5d\") " pod="openshift-image-registry/node-ca-478xv"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710949    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-sysctl-conf\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710952    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1df53334-7b61-47d0-9787-1dc604c84896-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710976    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-hostroot\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710981    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ffef2c6f-fe5b-4b5e-9d95-e7eac9b4f8be-konnectivity-ca\") pod \"konnectivity-agent-sdmdp\" (UID: \"ffef2c6f-fe5b-4b5e-9d95-e7eac9b4f8be\") " pod="kube-system/konnectivity-agent-sdmdp"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.710988    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5da3f960-f4f8-4301-89ef-1823ec93bff5-multus-daemon-config\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711017    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-multus-socket-dir-parent\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711042    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-log-socket\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711045    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-sys-fs\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711047    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711087    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711090    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5bfaaaff-fed0-456e-a757-8655bc43f26d-ovnkube-config\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711120    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs\") pod \"network-metrics-daemon-282m9\" (UID: \"87684818-77c6-45bc-bee8-61a206348549\") " pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711147    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-socket-dir\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711162    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-lib-modules\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.712493 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711175    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-var-lib-kubelet\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711224    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1df53334-7b61-47d0-9787-1dc604c84896-cnibin\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711253    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1df53334-7b61-47d0-9787-1dc604c84896-cni-binary-copy\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711266    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-run-ovn\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711284    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-system-cni-dir\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711299    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d96fa3e0-be40-4f37-99be-57261dce04d1-tmp-dir\") pod \"node-resolver-57l9p\" (UID: \"d96fa3e0-be40-4f37-99be-57261dce04d1\") " pod="openshift-dns/node-resolver-57l9p"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711308    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wthl\" (UniqueName: \"kubernetes.io/projected/5da3f960-f4f8-4301-89ef-1823ec93bff5-kube-api-access-5wthl\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711326    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1df53334-7b61-47d0-9787-1dc604c84896-system-cni-dir\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711337    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv77m\" (UniqueName: \"kubernetes.io/projected/1df53334-7b61-47d0-9787-1dc604c84896-kube-api-access-lv77m\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711376    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/73a1706e-dce9-475e-b553-642999cc42c0-dbus\") pod \"global-pull-secret-syncer-c8l7h\" (UID: \"73a1706e-dce9-475e-b553-642999cc42c0\") " pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711382    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d96fa3e0-be40-4f37-99be-57261dce04d1-hosts-file\") pod \"node-resolver-57l9p\" (UID: \"d96fa3e0-be40-4f37-99be-57261dce04d1\") " pod="openshift-dns/node-resolver-57l9p"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711411    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-cnibin\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711439    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-var-lib-kubelet\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711462    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-sysctl-conf\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711471    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5bfaaaff-fed0-456e-a757-8655bc43f26d-ovnkube-script-lib\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711485    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-socket-dir\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711499    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/366fd53e-0cd4-4fc9-8737-00383ad97707-tmp\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.713303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711523    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1df53334-7b61-47d0-9787-1dc604c84896-cnibin\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:33.711539    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711546    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-device-dir\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711577    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-var-lib-kubelet\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711595    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckj94\" (UniqueName: \"kubernetes.io/projected/d96fa3e0-be40-4f37-99be-57261dce04d1-kube-api-access-ckj94\") pod \"node-resolver-57l9p\" (UID: \"d96fa3e0-be40-4f37-99be-57261dce04d1\") " pod="openshift-dns/node-resolver-57l9p"
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:33.711612    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs podName:87684818-77c6-45bc-bee8-61a206348549 nodeName:}" failed. No retries permitted until 2026-09-16 18:39:34.211590755 +0000 UTC m=+1.999154889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs") pod "network-metrics-daemon-282m9" (UID: "87684818-77c6-45bc-bee8-61a206348549") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711621    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-run-ovn\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711375    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-multus-socket-dir-parent\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711670    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ffef2c6f-fe5b-4b5e-9d95-e7eac9b4f8be-konnectivity-ca\") pod \"konnectivity-agent-sdmdp\" (UID: \"ffef2c6f-fe5b-4b5e-9d95-e7eac9b4f8be\") " pod="kube-system/konnectivity-agent-sdmdp"
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711693    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-cnibin\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711711    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5bfaaaff-fed0-456e-a757-8655bc43f26d-ovnkube-config\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711737    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-var-lib-kubelet\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711806    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fae250ed-5b59-4388-af13-fcc8d0446e5d-serviceca\") pod \"node-ca-478xv\" (UID: \"fae250ed-5b59-4388-af13-fcc8d0446e5d\") " pod="openshift-image-registry/node-ca-478xv"
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711838    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7233615e-7ceb-4d7b-86d9-5b30e3b85b26-iptables-alerter-script\") pod \"iptables-alerter-h5qsz\" (UID: \"7233615e-7ceb-4d7b-86d9-5b30e3b85b26\") " pod="openshift-network-operator/iptables-alerter-h5qsz"
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711852    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-device-dir\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711865    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-var-lib-cni-multus\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711908    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-kubelet\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.713916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711922    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d96fa3e0-be40-4f37-99be-57261dce04d1-hosts-file\") pod \"node-resolver-57l9p\" (UID: \"d96fa3e0-be40-4f37-99be-57261dce04d1\") " pod="openshift-dns/node-resolver-57l9p"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711929    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-run-systemd\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711966    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-run-systemd\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.711967    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-sys\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712002    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-host\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712006    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-sys\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712015    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-kubelet\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712024    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5da3f960-f4f8-4301-89ef-1823ec93bff5-cni-binary-copy\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712047    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5bfaaaff-fed0-456e-a757-8655bc43f26d-env-overrides\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712056    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-var-lib-cni-multus\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-systemd\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712091    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-var-lib-cni-bin\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712108    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-host\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712114    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-etc-kubernetes\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712146    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-etc-kubernetes\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712146    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-run-ovn-kubernetes\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712177    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret\") pod \"global-pull-secret-syncer-c8l7h\" (UID: \"73a1706e-dce9-475e-b553-642999cc42c0\") " pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712181    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-run-ovn-kubernetes\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.714455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712091    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1df53334-7b61-47d0-9787-1dc604c84896-cni-binary-copy\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712203    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpgjd\" (UniqueName: \"kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd\") pod \"network-check-target-wbfzv\" (UID: \"2aad88e5-b35d-4876-8a3b-c8b0b64f8570\") " pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712233    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-sysconfig\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712283    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-kubernetes\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712309    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-systemd-units\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712338    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zkfv\" (UniqueName: \"kubernetes.io/projected/87684818-77c6-45bc-bee8-61a206348549-kube-api-access-6zkfv\") pod \"network-metrics-daemon-282m9\" (UID: \"87684818-77c6-45bc-bee8-61a206348549\") " pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-modprobe-d\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712394    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7233615e-7ceb-4d7b-86d9-5b30e3b85b26-host-slash\") pod \"iptables-alerter-h5qsz\" (UID: \"7233615e-7ceb-4d7b-86d9-5b30e3b85b26\") " pod="openshift-network-operator/iptables-alerter-h5qsz"
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712423    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5bfaaaff-fed0-456e-a757-8655bc43f26d-ovnkube-script-lib\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712424    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dtrs\" (UniqueName: \"kubernetes.io/projected/7233615e-7ceb-4d7b-86d9-5b30e3b85b26-kube-api-access-5dtrs\") pod \"iptables-alerter-h5qsz\" (UID: \"7233615e-7ceb-4d7b-86d9-5b30e3b85b26\") " pod="openshift-network-operator/iptables-alerter-h5qsz"
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:33.712453    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712469    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fae250ed-5b59-4388-af13-fcc8d0446e5d-serviceca\") pod \"node-ca-478xv\" (UID: \"fae250ed-5b59-4388-af13-fcc8d0446e5d\") " pod="openshift-image-registry/node-ca-478xv"
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712469    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5bfaaaff-fed0-456e-a757-8655bc43f26d-env-overrides\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:33.712501    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret podName:73a1706e-dce9-475e-b553-642999cc42c0 nodeName:}" failed. No retries permitted until 2026-09-16 18:39:34.212482078 +0000 UTC m=+2.000046215 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret") pod "global-pull-secret-syncer-c8l7h" (UID: "73a1706e-dce9-475e-b553-642999cc42c0") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712499    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-run-multus-certs\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712540    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-slash\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712539    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-var-lib-cni-bin\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.714984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712500    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-systemd\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712570    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-kubernetes\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712577    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-slash\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712592    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-systemd-units\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712598    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-modprobe-d\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712606    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-sysconfig\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712624    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-node-log\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712639    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5da3f960-f4f8-4301-89ef-1823ec93bff5-host-run-multus-certs\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-cni-netd\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712678    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-node-log\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712692    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5bfaaaff-fed0-456e-a757-8655bc43f26d-host-cni-netd\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712685    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1df53334-7b61-47d0-9787-1dc604c84896-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712733    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-registration-dir\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712762    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5bfaaaff-fed0-456e-a757-8655bc43f26d-ovn-node-metrics-cert\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712788    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvqd7\" (UniqueName: \"kubernetes.io/projected/5bfaaaff-fed0-456e-a757-8655bc43f26d-kube-api-access-tvqd7\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712811    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71dda4a6-6301-4bf9-abb7-b318f57fe02e-registration-dir\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712813    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/73a1706e-dce9-475e-b553-642999cc42c0-kubelet-config\") pod \"global-pull-secret-syncer-c8l7h\" (UID: \"73a1706e-dce9-475e-b553-642999cc42c0\") " pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:33.715690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712861    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/73a1706e-dce9-475e-b553-642999cc42c0-kubelet-config\") pod \"global-pull-secret-syncer-c8l7h\" (UID: \"73a1706e-dce9-475e-b553-642999cc42c0\") " pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:33.716201 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712867    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ffef2c6f-fe5b-4b5e-9d95-e7eac9b4f8be-agent-certs\") pod \"konnectivity-agent-sdmdp\" (UID: \"ffef2c6f-fe5b-4b5e-9d95-e7eac9b4f8be\") " pod="kube-system/konnectivity-agent-sdmdp"
Sep 16 18:39:33.716201 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.712935    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5da3f960-f4f8-4301-89ef-1823ec93bff5-cni-binary-copy\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.716201 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.713188    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1df53334-7b61-47d0-9787-1dc604c84896-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.716201 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.715657    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/366fd53e-0cd4-4fc9-8737-00383ad97707-etc-tuned\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.716201 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.715714    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/366fd53e-0cd4-4fc9-8737-00383ad97707-tmp\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.716201 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.715792    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5bfaaaff-fed0-456e-a757-8655bc43f26d-ovn-node-metrics-cert\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.716201 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.715905    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ffef2c6f-fe5b-4b5e-9d95-e7eac9b4f8be-agent-certs\") pod \"konnectivity-agent-sdmdp\" (UID: \"ffef2c6f-fe5b-4b5e-9d95-e7eac9b4f8be\") " pod="kube-system/konnectivity-agent-sdmdp"
Sep 16 18:39:33.720502 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.720478    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6792\" (UniqueName: \"kubernetes.io/projected/71dda4a6-6301-4bf9-abb7-b318f57fe02e-kube-api-access-d6792\") pod \"aws-ebs-csi-driver-node-zhft6\" (UID: \"71dda4a6-6301-4bf9-abb7-b318f57fe02e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.720602 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.720482    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcc9v\" (UniqueName: \"kubernetes.io/projected/366fd53e-0cd4-4fc9-8737-00383ad97707-kube-api-access-kcc9v\") pod \"tuned-v8fzr\" (UID: \"366fd53e-0cd4-4fc9-8737-00383ad97707\") " pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:33.721878 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.721449    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-468sz\" (UniqueName: \"kubernetes.io/projected/fae250ed-5b59-4388-af13-fcc8d0446e5d-kube-api-access-468sz\") pod \"node-ca-478xv\" (UID: \"fae250ed-5b59-4388-af13-fcc8d0446e5d\") " pod="openshift-image-registry/node-ca-478xv"
Sep 16 18:39:33.726043 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:33.726023    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:39:33.726141 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:33.726049    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:39:33.726141 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:33.726064    2578 projected.go:196] Error preparing data for projected volume kube-api-access-lpgjd for pod openshift-network-diagnostics/network-check-target-wbfzv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:39:33.726141 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:33.726123    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd podName:2aad88e5-b35d-4876-8a3b-c8b0b64f8570 nodeName:}" failed. No retries permitted until 2026-09-16 18:39:34.226106096 +0000 UTC m=+2.013670235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lpgjd" (UniqueName: "kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd") pod "network-check-target-wbfzv" (UID: "2aad88e5-b35d-4876-8a3b-c8b0b64f8570") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:39:33.728045 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.728014    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckj94\" (UniqueName: \"kubernetes.io/projected/d96fa3e0-be40-4f37-99be-57261dce04d1-kube-api-access-ckj94\") pod \"node-resolver-57l9p\" (UID: \"d96fa3e0-be40-4f37-99be-57261dce04d1\") " pod="openshift-dns/node-resolver-57l9p"
Sep 16 18:39:33.728212 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.728190    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wthl\" (UniqueName: \"kubernetes.io/projected/5da3f960-f4f8-4301-89ef-1823ec93bff5-kube-api-access-5wthl\") pod \"multus-qx9nj\" (UID: \"5da3f960-f4f8-4301-89ef-1823ec93bff5\") " pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.728564 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.728545    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvqd7\" (UniqueName: \"kubernetes.io/projected/5bfaaaff-fed0-456e-a757-8655bc43f26d-kube-api-access-tvqd7\") pod \"ovnkube-node-gbxkd\" (UID: \"5bfaaaff-fed0-456e-a757-8655bc43f26d\") " pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.728638 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.728618    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zkfv\" (UniqueName: \"kubernetes.io/projected/87684818-77c6-45bc-bee8-61a206348549-kube-api-access-6zkfv\") pod \"network-metrics-daemon-282m9\" (UID: \"87684818-77c6-45bc-bee8-61a206348549\") " pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:33.730085 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.730070    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv77m\" (UniqueName: \"kubernetes.io/projected/1df53334-7b61-47d0-9787-1dc604c84896-kube-api-access-lv77m\") pod \"multus-additional-cni-plugins-nsrzq\" (UID: \"1df53334-7b61-47d0-9787-1dc604c84896\") " pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:33.813813 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.813774    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7233615e-7ceb-4d7b-86d9-5b30e3b85b26-host-slash\") pod \"iptables-alerter-h5qsz\" (UID: \"7233615e-7ceb-4d7b-86d9-5b30e3b85b26\") " pod="openshift-network-operator/iptables-alerter-h5qsz"
Sep 16 18:39:33.813813 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.813808    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dtrs\" (UniqueName: \"kubernetes.io/projected/7233615e-7ceb-4d7b-86d9-5b30e3b85b26-kube-api-access-5dtrs\") pod \"iptables-alerter-h5qsz\" (UID: \"7233615e-7ceb-4d7b-86d9-5b30e3b85b26\") " pod="openshift-network-operator/iptables-alerter-h5qsz"
Sep 16 18:39:33.814031 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.813855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7233615e-7ceb-4d7b-86d9-5b30e3b85b26-iptables-alerter-script\") pod \"iptables-alerter-h5qsz\" (UID: \"7233615e-7ceb-4d7b-86d9-5b30e3b85b26\") " pod="openshift-network-operator/iptables-alerter-h5qsz"
Sep 16 18:39:33.814031 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.813889    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7233615e-7ceb-4d7b-86d9-5b30e3b85b26-host-slash\") pod \"iptables-alerter-h5qsz\" (UID: \"7233615e-7ceb-4d7b-86d9-5b30e3b85b26\") " pod="openshift-network-operator/iptables-alerter-h5qsz"
Sep 16 18:39:33.814366 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.814349    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7233615e-7ceb-4d7b-86d9-5b30e3b85b26-iptables-alerter-script\") pod \"iptables-alerter-h5qsz\" (UID: \"7233615e-7ceb-4d7b-86d9-5b30e3b85b26\") " pod="openshift-network-operator/iptables-alerter-h5qsz"
Sep 16 18:39:33.823104 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.823074    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dtrs\" (UniqueName: \"kubernetes.io/projected/7233615e-7ceb-4d7b-86d9-5b30e3b85b26-kube-api-access-5dtrs\") pod \"iptables-alerter-h5qsz\" (UID: \"7233615e-7ceb-4d7b-86d9-5b30e3b85b26\") " pod="openshift-network-operator/iptables-alerter-h5qsz"
Sep 16 18:39:33.913922 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.913832    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sdmdp"
Sep 16 18:39:33.923205 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:33.923171    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffef2c6f_fe5b_4b5e_9d95_e7eac9b4f8be.slice/crio-5039ed9a8e02853f70339fc60e94f76a00182105405e652e2b05dad5817e43aa WatchSource:0}: Error finding container 5039ed9a8e02853f70339fc60e94f76a00182105405e652e2b05dad5817e43aa: Status 404 returned error can't find the container with id 5039ed9a8e02853f70339fc60e94f76a00182105405e652e2b05dad5817e43aa
Sep 16 18:39:33.932373 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.932348    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6"
Sep 16 18:39:33.939993 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:33.939829    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71dda4a6_6301_4bf9_abb7_b318f57fe02e.slice/crio-228745ea8116e8c6d855d2fdfd2f292ca2d3d8f0cfd90bbff1c2a78a389e449d WatchSource:0}: Error finding container 228745ea8116e8c6d855d2fdfd2f292ca2d3d8f0cfd90bbff1c2a78a389e449d: Status 404 returned error can't find the container with id 228745ea8116e8c6d855d2fdfd2f292ca2d3d8f0cfd90bbff1c2a78a389e449d
Sep 16 18:39:33.941714 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.941694    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-57l9p"
Sep 16 18:39:33.949718 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:33.949693    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd96fa3e0_be40_4f37_99be_57261dce04d1.slice/crio-42cd2d8a8d1c778803387fc5020f0c2d9345a347160eb5f0e2da0a05dd0a42a8 WatchSource:0}: Error finding container 42cd2d8a8d1c778803387fc5020f0c2d9345a347160eb5f0e2da0a05dd0a42a8: Status 404 returned error can't find the container with id 42cd2d8a8d1c778803387fc5020f0c2d9345a347160eb5f0e2da0a05dd0a42a8
Sep 16 18:39:33.953973 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.953948    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-478xv"
Sep 16 18:39:33.961362 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:33.961337    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae250ed_5b59_4388_af13_fcc8d0446e5d.slice/crio-3ea438491f42d33a012e6e00d69038023eb42e57dbd7d9b44c9cfff8468b66a7 WatchSource:0}: Error finding container 3ea438491f42d33a012e6e00d69038023eb42e57dbd7d9b44c9cfff8468b66a7: Status 404 returned error can't find the container with id 3ea438491f42d33a012e6e00d69038023eb42e57dbd7d9b44c9cfff8468b66a7
Sep 16 18:39:33.973924 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.973906    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qx9nj"
Sep 16 18:39:33.981497 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:33.981467    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da3f960_f4f8_4301_89ef_1823ec93bff5.slice/crio-111a31c88497a8fb17d1f82ab79a2829858afdcd9d54448f720207894dbee4ec WatchSource:0}: Error finding container 111a31c88497a8fb17d1f82ab79a2829858afdcd9d54448f720207894dbee4ec: Status 404 returned error can't find the container with id 111a31c88497a8fb17d1f82ab79a2829858afdcd9d54448f720207894dbee4ec
Sep 16 18:39:33.982268 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.982232    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:39:33.992017 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:33.991981    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bfaaaff_fed0_456e_a757_8655bc43f26d.slice/crio-e874eea7a4c0b6daf371e39e8158c061f8fb9fa68057b47c4c0bd2e9939d6183 WatchSource:0}: Error finding container e874eea7a4c0b6daf371e39e8158c061f8fb9fa68057b47c4c0bd2e9939d6183: Status 404 returned error can't find the container with id e874eea7a4c0b6daf371e39e8158c061f8fb9fa68057b47c4c0bd2e9939d6183
Sep 16 18:39:33.995173 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:33.995156    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-v8fzr"
Sep 16 18:39:34.001059 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.001033    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nsrzq"
Sep 16 18:39:34.002998 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:34.002976    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod366fd53e_0cd4_4fc9_8737_00383ad97707.slice/crio-6b23659feae1e8d16c92c9370baa61f935c8ea3e9080fccb79d9b122e04c5342 WatchSource:0}: Error finding container 6b23659feae1e8d16c92c9370baa61f935c8ea3e9080fccb79d9b122e04c5342: Status 404 returned error can't find the container with id 6b23659feae1e8d16c92c9370baa61f935c8ea3e9080fccb79d9b122e04c5342
Sep 16 18:39:34.007279 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.007258    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h5qsz"
Sep 16 18:39:34.009069 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:34.009050    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1df53334_7b61_47d0_9787_1dc604c84896.slice/crio-d30660b4e1729cc6fd2cf1c23f1bf210be0d765bd9967199d3deb8083462455f WatchSource:0}: Error finding container d30660b4e1729cc6fd2cf1c23f1bf210be0d765bd9967199d3deb8083462455f: Status 404 returned error can't find the container with id d30660b4e1729cc6fd2cf1c23f1bf210be0d765bd9967199d3deb8083462455f
Sep 16 18:39:34.015141 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:39:34.015120    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7233615e_7ceb_4d7b_86d9_5b30e3b85b26.slice/crio-c4e02ea6c4d1f499f8bde84513131bef090d57589fda030e859ffbad927345ef WatchSource:0}: Error finding container c4e02ea6c4d1f499f8bde84513131bef090d57589fda030e859ffbad927345ef: Status 404 returned error can't find the container with id c4e02ea6c4d1f499f8bde84513131bef090d57589fda030e859ffbad927345ef
Sep 16 18:39:34.216259 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.215605    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs\") pod \"network-metrics-daemon-282m9\" (UID: \"87684818-77c6-45bc-bee8-61a206348549\") " pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:34.216259 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.215665    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret\") pod \"global-pull-secret-syncer-c8l7h\" (UID: \"73a1706e-dce9-475e-b553-642999cc42c0\") " pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:34.216259 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:34.215814    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:39:34.216259 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:34.215864    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:39:34.216259 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:34.215879    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret podName:73a1706e-dce9-475e-b553-642999cc42c0 nodeName:}" failed. No retries permitted until 2026-09-16 18:39:35.215864597 +0000 UTC m=+3.003428732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret") pod "global-pull-secret-syncer-c8l7h" (UID: "73a1706e-dce9-475e-b553-642999cc42c0") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:39:34.216259 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:34.215953    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs podName:87684818-77c6-45bc-bee8-61a206348549 nodeName:}" failed. No retries permitted until 2026-09-16 18:39:35.21592848 +0000 UTC m=+3.003492628 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs") pod "network-metrics-daemon-282m9" (UID: "87684818-77c6-45bc-bee8-61a206348549") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:39:34.317208 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.316844    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpgjd\" (UniqueName: \"kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd\") pod \"network-check-target-wbfzv\" (UID: \"2aad88e5-b35d-4876-8a3b-c8b0b64f8570\") " pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:34.317208 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:34.317084    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:39:34.317208 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:34.317107    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:39:34.317208 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:34.317123    2578 projected.go:196] Error preparing data for projected volume kube-api-access-lpgjd for pod openshift-network-diagnostics/network-check-target-wbfzv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:39:34.318142 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:34.318100    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd podName:2aad88e5-b35d-4876-8a3b-c8b0b64f8570 nodeName:}" failed. No retries permitted until 2026-09-16 18:39:35.317169359 +0000 UTC m=+3.104733494 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpgjd" (UniqueName: "kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd") pod "network-check-target-wbfzv" (UID: "2aad88e5-b35d-4876-8a3b-c8b0b64f8570") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:39:34.517325 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.517228    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:39:34.653124 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.653063    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 18:34:34 +0000 UTC" deadline="2028-06-28 09:37:53.709129538 +0000 UTC"
Sep 16 18:39:34.653124 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.653123    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15614h58m19.056012659s"
Sep 16 18:39:34.740560 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.740455    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h5qsz" event={"ID":"7233615e-7ceb-4d7b-86d9-5b30e3b85b26","Type":"ContainerStarted","Data":"c4e02ea6c4d1f499f8bde84513131bef090d57589fda030e859ffbad927345ef"}
Sep 16 18:39:34.751301 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.751125    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd" event={"ID":"5bfaaaff-fed0-456e-a757-8655bc43f26d","Type":"ContainerStarted","Data":"e874eea7a4c0b6daf371e39e8158c061f8fb9fa68057b47c4c0bd2e9939d6183"}
Sep 16 18:39:34.767966 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.767832    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qx9nj" event={"ID":"5da3f960-f4f8-4301-89ef-1823ec93bff5","Type":"ContainerStarted","Data":"111a31c88497a8fb17d1f82ab79a2829858afdcd9d54448f720207894dbee4ec"}
Sep 16 18:39:34.775186 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.775092    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-57l9p" event={"ID":"d96fa3e0-be40-4f37-99be-57261dce04d1","Type":"ContainerStarted","Data":"42cd2d8a8d1c778803387fc5020f0c2d9345a347160eb5f0e2da0a05dd0a42a8"}
Sep 16 18:39:34.793053 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.792949    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sdmdp" event={"ID":"ffef2c6f-fe5b-4b5e-9d95-e7eac9b4f8be","Type":"ContainerStarted","Data":"5039ed9a8e02853f70339fc60e94f76a00182105405e652e2b05dad5817e43aa"}
Sep 16 18:39:34.820989 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.820566    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nsrzq" event={"ID":"1df53334-7b61-47d0-9787-1dc604c84896","Type":"ContainerStarted","Data":"d30660b4e1729cc6fd2cf1c23f1bf210be0d765bd9967199d3deb8083462455f"}
Sep 16 18:39:34.834280 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.834219    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v8fzr" event={"ID":"366fd53e-0cd4-4fc9-8737-00383ad97707","Type":"ContainerStarted","Data":"6b23659feae1e8d16c92c9370baa61f935c8ea3e9080fccb79d9b122e04c5342"}
Sep 16 18:39:34.844171 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.844129    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-478xv" event={"ID":"fae250ed-5b59-4388-af13-fcc8d0446e5d","Type":"ContainerStarted","Data":"3ea438491f42d33a012e6e00d69038023eb42e57dbd7d9b44c9cfff8468b66a7"}
Sep 16 18:39:34.851484 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:34.851440    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6" event={"ID":"71dda4a6-6301-4bf9-abb7-b318f57fe02e","Type":"ContainerStarted","Data":"228745ea8116e8c6d855d2fdfd2f292ca2d3d8f0cfd90bbff1c2a78a389e449d"}
Sep 16 18:39:35.225320 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:35.225282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs\") pod \"network-metrics-daemon-282m9\" (UID: \"87684818-77c6-45bc-bee8-61a206348549\") " pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:35.225715 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:35.225691    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret\") pod \"global-pull-secret-syncer-c8l7h\" (UID: \"73a1706e-dce9-475e-b553-642999cc42c0\") " pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:35.225996 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:35.225977    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:39:35.226232 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:35.226218    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret podName:73a1706e-dce9-475e-b553-642999cc42c0 nodeName:}" failed. No retries permitted until 2026-09-16 18:39:37.226195807 +0000 UTC m=+5.013759948 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret") pod "global-pull-secret-syncer-c8l7h" (UID: "73a1706e-dce9-475e-b553-642999cc42c0") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:39:35.226380 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:35.226360    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:39:35.226445 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:35.226422    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs podName:87684818-77c6-45bc-bee8-61a206348549 nodeName:}" failed. No retries permitted until 2026-09-16 18:39:37.226405278 +0000 UTC m=+5.013969426 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs") pod "network-metrics-daemon-282m9" (UID: "87684818-77c6-45bc-bee8-61a206348549") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:39:35.327114 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:35.327075    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpgjd\" (UniqueName: \"kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd\") pod \"network-check-target-wbfzv\" (UID: \"2aad88e5-b35d-4876-8a3b-c8b0b64f8570\") " pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:35.327341 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:35.327321    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:39:35.327413 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:35.327349    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:39:35.327413 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:35.327362    2578 projected.go:196] Error preparing data for projected volume kube-api-access-lpgjd for pod openshift-network-diagnostics/network-check-target-wbfzv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:39:35.327535 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:35.327486    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd podName:2aad88e5-b35d-4876-8a3b-c8b0b64f8570 nodeName:}" failed. No retries permitted until 2026-09-16 18:39:37.327407314 +0000 UTC m=+5.114971473 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpgjd" (UniqueName: "kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd") pod "network-check-target-wbfzv" (UID: "2aad88e5-b35d-4876-8a3b-c8b0b64f8570") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:39:35.492905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:35.492821    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:39:35.653430 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:35.653319    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 18:34:34 +0000 UTC" deadline="2028-02-22 04:30:09.892541615 +0000 UTC"
Sep 16 18:39:35.653430 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:35.653380    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12561h50m34.239166592s"
Sep 16 18:39:35.704453 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:35.703449    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:35.704453 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:35.703609    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:39:35.704453 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:35.704084    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:35.704453 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:35.704182    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:39:35.704453 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:35.704250    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:35.704453 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:35.704357    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:39:37.241945 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:37.241902    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret\") pod \"global-pull-secret-syncer-c8l7h\" (UID: \"73a1706e-dce9-475e-b553-642999cc42c0\") " pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:37.242450 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:37.242011    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs\") pod \"network-metrics-daemon-282m9\" (UID: \"87684818-77c6-45bc-bee8-61a206348549\") " pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:37.242450 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:37.242198    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:39:37.242450 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:37.242288    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs podName:87684818-77c6-45bc-bee8-61a206348549 nodeName:}" failed. No retries permitted until 2026-09-16 18:39:41.242262213 +0000 UTC m=+9.029826349 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs") pod "network-metrics-daemon-282m9" (UID: "87684818-77c6-45bc-bee8-61a206348549") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:39:37.242724 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:37.242677    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:39:37.242724 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:37.242722    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret podName:73a1706e-dce9-475e-b553-642999cc42c0 nodeName:}" failed. No retries permitted until 2026-09-16 18:39:41.242710646 +0000 UTC m=+9.030274781 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret") pod "global-pull-secret-syncer-c8l7h" (UID: "73a1706e-dce9-475e-b553-642999cc42c0") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:39:37.343563 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:37.342909    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpgjd\" (UniqueName: \"kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd\") pod \"network-check-target-wbfzv\" (UID: \"2aad88e5-b35d-4876-8a3b-c8b0b64f8570\") " pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:37.343563 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:37.343076    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:39:37.343563 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:37.343097    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:39:37.343563 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:37.343112    2578 projected.go:196] Error preparing data for projected volume kube-api-access-lpgjd for pod openshift-network-diagnostics/network-check-target-wbfzv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:39:37.343563 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:37.343182    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd podName:2aad88e5-b35d-4876-8a3b-c8b0b64f8570 nodeName:}" failed. No retries permitted until 2026-09-16 18:39:41.343162229 +0000 UTC m=+9.130726370 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpgjd" (UniqueName: "kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd") pod "network-check-target-wbfzv" (UID: "2aad88e5-b35d-4876-8a3b-c8b0b64f8570") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:39:37.702771 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:37.702728    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:37.702956 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:37.702876    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:39:37.702956 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:37.702738    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:37.703092 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:37.702985    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:39:37.703092 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:37.702728    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:37.703092 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:37.703078    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:39:39.703683 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:39.703340    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:39.703683 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:39.703471    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:39:39.703683 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:39.703481    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:39.703683 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:39.703508    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:39.703683 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:39.703578    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:39:39.703683 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:39.703652    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:39:41.279555 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:41.279510    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret\") pod \"global-pull-secret-syncer-c8l7h\" (UID: \"73a1706e-dce9-475e-b553-642999cc42c0\") " pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:41.280041 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:41.279615    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs\") pod \"network-metrics-daemon-282m9\" (UID: \"87684818-77c6-45bc-bee8-61a206348549\") " pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:41.280041 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:41.279678    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:39:41.280041 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:41.279732    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:39:41.280041 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:41.279758    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret podName:73a1706e-dce9-475e-b553-642999cc42c0 nodeName:}" failed. No retries permitted until 2026-09-16 18:39:49.279736968 +0000 UTC m=+17.067301118 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret") pod "global-pull-secret-syncer-c8l7h" (UID: "73a1706e-dce9-475e-b553-642999cc42c0") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:39:41.280041 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:41.279784    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs podName:87684818-77c6-45bc-bee8-61a206348549 nodeName:}" failed. No retries permitted until 2026-09-16 18:39:49.279768179 +0000 UTC m=+17.067332317 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs") pod "network-metrics-daemon-282m9" (UID: "87684818-77c6-45bc-bee8-61a206348549") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:39:41.380740 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:41.380643    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpgjd\" (UniqueName: \"kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd\") pod \"network-check-target-wbfzv\" (UID: \"2aad88e5-b35d-4876-8a3b-c8b0b64f8570\") " pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:41.380896 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:41.380799    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:39:41.380896 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:41.380819    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:39:41.380896 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:41.380832    2578 projected.go:196] Error preparing data for projected volume kube-api-access-lpgjd for pod openshift-network-diagnostics/network-check-target-wbfzv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:39:41.380896 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:41.380896    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd podName:2aad88e5-b35d-4876-8a3b-c8b0b64f8570 nodeName:}" failed. No retries permitted until 2026-09-16 18:39:49.380878079 +0000 UTC m=+17.168442214 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpgjd" (UniqueName: "kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd") pod "network-check-target-wbfzv" (UID: "2aad88e5-b35d-4876-8a3b-c8b0b64f8570") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:39:41.703291 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:41.702885    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:41.703291 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:41.702872    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:41.703291 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:41.703021    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:39:41.703291 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:41.703059    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:41.703291 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:41.703154    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:39:41.703291 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:41.703270    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:39:43.703187 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:43.703135    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:43.703734 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:43.703197    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:43.703734 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:43.703135    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:43.703734 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:43.703297    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:39:43.703734 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:43.703374    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:39:43.703734 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:43.703451    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:39:45.703095 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:45.703008    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:45.703095 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:45.703046    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:45.703666 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:45.703120    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:45.703666 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:45.703138    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:39:45.703666 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:45.703209    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:39:45.703666 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:45.703297    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:39:47.703351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:47.703311    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:47.703786 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:47.703311    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:47.703786 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:47.703439    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:39:47.703786 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:47.703536    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:39:47.703786 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:47.703311    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:47.703786 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:47.703643    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:39:49.342990 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:49.342946    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret\") pod \"global-pull-secret-syncer-c8l7h\" (UID: \"73a1706e-dce9-475e-b553-642999cc42c0\") " pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:49.343481 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:49.343040    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs\") pod \"network-metrics-daemon-282m9\" (UID: \"87684818-77c6-45bc-bee8-61a206348549\") " pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:49.343481 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:49.343116    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:39:49.343481 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:49.343149    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:39:49.343481 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:49.343217    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret podName:73a1706e-dce9-475e-b553-642999cc42c0 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:05.343195315 +0000 UTC m=+33.130759448 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret") pod "global-pull-secret-syncer-c8l7h" (UID: "73a1706e-dce9-475e-b553-642999cc42c0") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:39:49.343481 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:49.343239    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs podName:87684818-77c6-45bc-bee8-61a206348549 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:05.343228786 +0000 UTC m=+33.130792927 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs") pod "network-metrics-daemon-282m9" (UID: "87684818-77c6-45bc-bee8-61a206348549") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:39:49.444289 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:49.444249    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpgjd\" (UniqueName: \"kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd\") pod \"network-check-target-wbfzv\" (UID: \"2aad88e5-b35d-4876-8a3b-c8b0b64f8570\") " pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:49.444487 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:49.444439    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:39:49.444487 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:49.444471    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:39:49.444487 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:49.444483    2578 projected.go:196] Error preparing data for projected volume kube-api-access-lpgjd for pod openshift-network-diagnostics/network-check-target-wbfzv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:39:49.444641 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:49.444541    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd podName:2aad88e5-b35d-4876-8a3b-c8b0b64f8570 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:05.444522718 +0000 UTC m=+33.232086851 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpgjd" (UniqueName: "kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd") pod "network-check-target-wbfzv" (UID: "2aad88e5-b35d-4876-8a3b-c8b0b64f8570") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:39:49.703399 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:49.703361    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:49.703598 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:49.703361    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:49.703598 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:49.703477    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:39:49.703598 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:49.703366    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:49.703598 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:49.703565    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:39:49.703814 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:49.703637    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:39:51.702497 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:51.702467    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:51.702497 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:51.702486    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:51.703131 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:51.702475    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:51.703131 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:51.702586    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:39:51.703131 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:51.702670    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:39:51.703131 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:51.702777    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:39:53.703305 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.702999    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:53.703975 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.703000    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:53.703975 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:53.703386    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:39:53.703975 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.703034    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:53.703975 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:53.703474    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:39:53.703975 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:53.703546    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:39:53.904167 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.904134    2578 generic.go:356] "Generic (PLEG): container finished" podID="1df53334-7b61-47d0-9787-1dc604c84896" containerID="b62e3a3ddf7da4fc2dca283c384523ec926a86c46c1e5f30f24edeca8e628fbd" exitCode=0
Sep 16 18:39:53.904368 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.904228    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nsrzq" event={"ID":"1df53334-7b61-47d0-9787-1dc604c84896","Type":"ContainerDied","Data":"b62e3a3ddf7da4fc2dca283c384523ec926a86c46c1e5f30f24edeca8e628fbd"}
Sep 16 18:39:53.905921 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.905890    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-v8fzr" event={"ID":"366fd53e-0cd4-4fc9-8737-00383ad97707","Type":"ContainerStarted","Data":"d5fe05f293d22d28f076e5e95208efbcd11cfc71f87b17ccefd3e088cfbed91f"}
Sep 16 18:39:53.907521 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.907469    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-478xv" event={"ID":"fae250ed-5b59-4388-af13-fcc8d0446e5d","Type":"ContainerStarted","Data":"929514f4952f7f022e6c485ce0548d140c7681eb506abac63dba4f42cdb4ca13"}
Sep 16 18:39:53.908982 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.908953    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6" event={"ID":"71dda4a6-6301-4bf9-abb7-b318f57fe02e","Type":"ContainerStarted","Data":"597f5a15571e55cac8d1d38bd852604d38935f77021f165695838d41f77ab468"}
Sep 16 18:39:53.910369 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.910350    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-234.ec2.internal" event={"ID":"41ec96924e9815932b677eedb2f521c7","Type":"ContainerStarted","Data":"e6991d38cb5a45d1fa9f3e0ba318bcd173f73335e53ced2c05c67c1486d6d970"}
Sep 16 18:39:53.911847 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.911827    2578 generic.go:356] "Generic (PLEG): container finished" podID="b8171f0e1a50a19cc3561ab6c613a155" containerID="8d5ad85be95ac77279613a6b71bf5b21a383e3ece685506c279390b13ac57fe3" exitCode=0
Sep 16 18:39:53.911922 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.911891    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal" event={"ID":"b8171f0e1a50a19cc3561ab6c613a155","Type":"ContainerDied","Data":"8d5ad85be95ac77279613a6b71bf5b21a383e3ece685506c279390b13ac57fe3"}
Sep 16 18:39:53.914768 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.914751    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 18:39:53.915103 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.915080    2578 generic.go:356] "Generic (PLEG): container finished" podID="5bfaaaff-fed0-456e-a757-8655bc43f26d" containerID="d531e69c00d421acdcd70f3d8debd49144f5859e553e8e348ed3faf4e6bc98e1" exitCode=1
Sep 16 18:39:53.915160 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.915136    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd" event={"ID":"5bfaaaff-fed0-456e-a757-8655bc43f26d","Type":"ContainerStarted","Data":"12bafaddaf5009bbe453a4dfd6b54eaf34792b40d7f6aee0c22ece327183fe1b"}
Sep 16 18:39:53.915226 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.915208    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd" event={"ID":"5bfaaaff-fed0-456e-a757-8655bc43f26d","Type":"ContainerStarted","Data":"f9722d38829573cfd59e4d0684878b675e6206dfc3e344cf32e406188b928ba1"}
Sep 16 18:39:53.915270 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.915252    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd" event={"ID":"5bfaaaff-fed0-456e-a757-8655bc43f26d","Type":"ContainerStarted","Data":"e7bbccc803e1f1ce3e846e6e051a621103f9cbf16321733e1f008066f8ca52ca"}
Sep 16 18:39:53.915307 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.915291    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd" event={"ID":"5bfaaaff-fed0-456e-a757-8655bc43f26d","Type":"ContainerStarted","Data":"e1bb38c73370655603394e7e7839a5012d12a75a03b36a5ce4635e7ac255b8ab"}
Sep 16 18:39:53.915341 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.915320    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd" event={"ID":"5bfaaaff-fed0-456e-a757-8655bc43f26d","Type":"ContainerDied","Data":"d531e69c00d421acdcd70f3d8debd49144f5859e553e8e348ed3faf4e6bc98e1"}
Sep 16 18:39:53.915377 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.915351    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd" event={"ID":"5bfaaaff-fed0-456e-a757-8655bc43f26d","Type":"ContainerStarted","Data":"5cd6468a565324a0b116b01777f21881f82622d9f15003c25753ee5e3b008713"}
Sep 16 18:39:53.919774 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.919749    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qx9nj" event={"ID":"5da3f960-f4f8-4301-89ef-1823ec93bff5","Type":"ContainerStarted","Data":"29c00bcddd60b5fad52efa78743c7530f6bb976408fa743d2e2763e1f1d2a8b4"}
Sep 16 18:39:53.921192 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.921150    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-57l9p" event={"ID":"d96fa3e0-be40-4f37-99be-57261dce04d1","Type":"ContainerStarted","Data":"5e0ee62f1c60fb10a58e7efb338669aa6cf890b40a0565994a83bd32c787b701"}
Sep 16 18:39:53.922645 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.922627    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sdmdp" event={"ID":"ffef2c6f-fe5b-4b5e-9d95-e7eac9b4f8be","Type":"ContainerStarted","Data":"6103959a4adf59fffbc6b7dc48d4d361483baf1743185e7d99ba12a8ed6dd3fc"}
Sep 16 18:39:53.981806 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:53.981747    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-v8fzr" podStartSLOduration=1.95942615 podStartE2EDuration="20.981728371s" totalImagesPullingTime="19.022302221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:39:53.980959874 +0000 UTC m=+21.768524048" watchObservedRunningTime="2026-09-16 18:39:53.981728371 +0000 UTC m=+21.769292526"
Sep 16 18:39:54.021372 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:54.021309    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sdmdp" podStartSLOduration=9.58831181 podStartE2EDuration="21.021292117s" totalImagesPullingTime="11.432980307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:39:54.020948118 +0000 UTC m=+21.808512273" watchObservedRunningTime="2026-09-16 18:39:54.021292117 +0000 UTC m=+21.808856271"
Sep 16 18:39:54.036563 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:54.036519    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-478xv" podStartSLOduration=2.024720105 podStartE2EDuration="21.036503704s" totalImagesPullingTime="19.011783599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:39:54.036321749 +0000 UTC m=+21.823885903" watchObservedRunningTime="2026-09-16 18:39:54.036503704 +0000 UTC m=+21.824067859"
Sep 16 18:39:54.077455 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:54.077413    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qx9nj" podStartSLOduration=1.997016015 podStartE2EDuration="21.077394912s" totalImagesPullingTime="19.080378897s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:39:54.057660319 +0000 UTC m=+21.845224473" watchObservedRunningTime="2026-09-16 18:39:54.077394912 +0000 UTC m=+21.864959068"
Sep 16 18:39:54.077781 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:54.077753    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-234.ec2.internal" podStartSLOduration=21.077745423 podStartE2EDuration="21.077745423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:39:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:39:54.07701015 +0000 UTC m=+21.864574305" watchObservedRunningTime="2026-09-16 18:39:54.077745423 +0000 UTC m=+21.865309577"
Sep 16 18:39:54.927534 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:54.927504    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h5qsz" event={"ID":"7233615e-7ceb-4d7b-86d9-5b30e3b85b26","Type":"ContainerStarted","Data":"de667042843e06b49a527d8e73aa0c2fc96bbfccabe7eed2ef3ca29d94ffa2e0"}
Sep 16 18:39:54.929417 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:54.929389    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal" event={"ID":"b8171f0e1a50a19cc3561ab6c613a155","Type":"ContainerStarted","Data":"0cef53d9435e4f2dee7b390c6bd2819fe652fae6f3bad9bbb06be6ef0a48545b"}
Sep 16 18:39:54.944678 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:54.944635    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-57l9p" podStartSLOduration=2.867450137 podStartE2EDuration="21.944614419s" totalImagesPullingTime="19.077164282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:39:54.102872616 +0000 UTC m=+21.890436797" watchObservedRunningTime="2026-09-16 18:39:54.944614419 +0000 UTC m=+22.732178574"
Sep 16 18:39:54.945260 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:54.945230    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h5qsz" podStartSLOduration=2.986859541 podStartE2EDuration="21.945220645s" totalImagesPullingTime="18.958361104s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:39:54.944730445 +0000 UTC m=+22.732294628" watchObservedRunningTime="2026-09-16 18:39:54.945220645 +0000 UTC m=+22.732784799"
Sep 16 18:39:54.997780 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:54.997752    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 18:39:55.661491 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:55.661388    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T18:39:54.99777357Z","UUID":"5bb44ce0-b596-44a9-9e3f-e04022594667","Handler":null,"Name":"","Endpoint":""}
Sep 16 18:39:55.662887 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:55.662867    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 18:39:55.663001 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:55.662896    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 18:39:55.703109 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:55.703080    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:55.703306 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:55.703084    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:55.703306 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:55.703224    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:39:55.703306 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:55.703083    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:55.703306 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:55.703272    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:39:55.703512 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:55.703337    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:39:55.932951 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:55.932925    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6" event={"ID":"71dda4a6-6301-4bf9-abb7-b318f57fe02e","Type":"ContainerStarted","Data":"7c94a076604137c705f17dc32f16e0cb331e9463c6d49747e053dfcc71199f17"}
Sep 16 18:39:56.758890 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:56.758845    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sdmdp"
Sep 16 18:39:56.759510 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:56.759482    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sdmdp"
Sep 16 18:39:56.778759 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:56.778712    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-234.ec2.internal" podStartSLOduration=23.778695864 podStartE2EDuration="23.778695864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:39:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:39:54.965958091 +0000 UTC m=+22.753522252" watchObservedRunningTime="2026-09-16 18:39:56.778695864 +0000 UTC m=+24.566260019"
Sep 16 18:39:56.937273 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:56.937235    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6" event={"ID":"71dda4a6-6301-4bf9-abb7-b318f57fe02e","Type":"ContainerStarted","Data":"e7c9452718912228f3b34f9ee0e690553329ec10c1040b3b7deabb3ac802fe10"}
Sep 16 18:39:56.940413 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:56.940392    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 18:39:56.940813 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:56.940793    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd" event={"ID":"5bfaaaff-fed0-456e-a757-8655bc43f26d","Type":"ContainerStarted","Data":"aafccc4fb83d91d54459d168d0a88432528066f19677720e54193c7e8028dc97"}
Sep 16 18:39:56.956496 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:56.956453    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zhft6" podStartSLOduration=2.503970667 podStartE2EDuration="23.956437597s" totalImagesPullingTime="21.45246693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:39:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:39:56.955996901 +0000 UTC m=+24.743561058" watchObservedRunningTime="2026-09-16 18:39:56.956437597 +0000 UTC m=+24.744001775"
Sep 16 18:39:57.702689 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:57.702658    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:57.702689 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:57.702683    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:57.702946 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:57.702705    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:57.702946 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:57.702779    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:39:57.702946 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:57.702873    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:39:57.703092 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:57.702947    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:39:58.085947 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:58.085820    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sdmdp"
Sep 16 18:39:58.086501 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:58.086025    2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 18:39:58.086710 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:58.086504    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sdmdp"
Sep 16 18:39:58.949584 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:58.949408    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 18:39:58.950016 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:58.949980    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd" event={"ID":"5bfaaaff-fed0-456e-a757-8655bc43f26d","Type":"ContainerStarted","Data":"e9d4a3c3dcad0ee6c9bd026630ce358cf82f744e1955c5ae048a8c8cd42638bf"}
Sep 16 18:39:58.950537 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:58.950519    2578 scope.go:117] "RemoveContainer" containerID="d531e69c00d421acdcd70f3d8debd49144f5859e553e8e348ed3faf4e6bc98e1"
Sep 16 18:39:59.702780 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:59.702737    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:39:59.702780 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:59.702776    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:39:59.703231 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:59.702783    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:39:59.703231 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:59.702852    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:39:59.703231 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:59.702948    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:39:59.703231 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:39:59.703037    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:39:59.987423 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:39:59.987347    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:40:00.240625 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:00.240555    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-282m9"]
Sep 16 18:40:00.240749 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:00.240692    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:40:00.240834 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:00.240816    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:40:00.243227 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:00.243201    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wbfzv"]
Sep 16 18:40:00.243350 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:00.243288    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:40:00.243390 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:00.243370    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:40:00.254209 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:00.254165    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c8l7h"]
Sep 16 18:40:00.254325 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:00.254255    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:40:00.254511 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:00.254319    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:40:00.958315 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:00.958287    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 18:40:00.958932 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:00.958675    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd" event={"ID":"5bfaaaff-fed0-456e-a757-8655bc43f26d","Type":"ContainerStarted","Data":"22bc04654fb447155592b356796d21f354c6328a868bd7cde78f1a14bc76b7af"}
Sep 16 18:40:00.958932 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:00.958906    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:40:00.959065 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:00.958995    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:40:00.960999 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:00.960973    2578 generic.go:356] "Generic (PLEG): container finished" podID="1df53334-7b61-47d0-9787-1dc604c84896" containerID="bb3a57fb4246c60b82d64e1840621320274ef6f0f66b59d5af3b38ac2c1e284b" exitCode=0
Sep 16 18:40:00.961119 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:00.961012    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nsrzq" event={"ID":"1df53334-7b61-47d0-9787-1dc604c84896","Type":"ContainerDied","Data":"bb3a57fb4246c60b82d64e1840621320274ef6f0f66b59d5af3b38ac2c1e284b"}
Sep 16 18:40:00.974793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:00.974770    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:40:00.974926 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:00.974904    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:40:00.995219 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:00.995161    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd" podStartSLOduration=8.829130658 podStartE2EDuration="27.995147054s" totalImagesPullingTime="19.166016396s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:00.994138617 +0000 UTC m=+28.781702773" watchObservedRunningTime="2026-09-16 18:40:00.995147054 +0000 UTC m=+28.782711200"
Sep 16 18:40:01.702975 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:01.702940    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:40:01.703141 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:01.702940    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:40:01.703141 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:01.703060    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:40:01.703141 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:01.702940    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:40:01.703325 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:01.703139    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:40:01.703325 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:01.703266    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:40:02.968091 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:02.967914    2578 generic.go:356] "Generic (PLEG): container finished" podID="1df53334-7b61-47d0-9787-1dc604c84896" containerID="e5173b6634e9c44539a5173a4a62a6baf1ca709419a14596ac12b96988a67e56" exitCode=0
Sep 16 18:40:02.968622 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:02.968003    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nsrzq" event={"ID":"1df53334-7b61-47d0-9787-1dc604c84896","Type":"ContainerDied","Data":"e5173b6634e9c44539a5173a4a62a6baf1ca709419a14596ac12b96988a67e56"}
Sep 16 18:40:03.703280 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:03.703241    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:40:03.703280 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:03.703265    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:40:03.703280 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:03.703255    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:40:03.703565 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:03.703365    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:40:03.703565 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:03.703428    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:40:03.703565 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:03.703508    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:40:04.977615 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:04.977582    2578 generic.go:356] "Generic (PLEG): container finished" podID="1df53334-7b61-47d0-9787-1dc604c84896" containerID="f4d2197db68dc2c58d2589b7057a0a20ed2d24965b756c4261497e3dd5979b1c" exitCode=0
Sep 16 18:40:04.978004 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:04.977646    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nsrzq" event={"ID":"1df53334-7b61-47d0-9787-1dc604c84896","Type":"ContainerDied","Data":"f4d2197db68dc2c58d2589b7057a0a20ed2d24965b756c4261497e3dd5979b1c"}
Sep 16 18:40:05.359946 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:05.359920    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret\") pod \"global-pull-secret-syncer-c8l7h\" (UID: \"73a1706e-dce9-475e-b553-642999cc42c0\") " pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:40:05.360089 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:05.359975    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs\") pod \"network-metrics-daemon-282m9\" (UID: \"87684818-77c6-45bc-bee8-61a206348549\") " pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:40:05.360089 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:05.360058    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:40:05.360151 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:05.360113    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs podName:87684818-77c6-45bc-bee8-61a206348549 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:37.360098832 +0000 UTC m=+65.147662970 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs") pod "network-metrics-daemon-282m9" (UID: "87684818-77c6-45bc-bee8-61a206348549") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:40:05.360151 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:05.360058    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:40:05.360243 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:05.360196    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret podName:73a1706e-dce9-475e-b553-642999cc42c0 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:37.360164888 +0000 UTC m=+65.147729021 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret") pod "global-pull-secret-syncer-c8l7h" (UID: "73a1706e-dce9-475e-b553-642999cc42c0") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:40:05.461048 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:05.461019    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpgjd\" (UniqueName: \"kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd\") pod \"network-check-target-wbfzv\" (UID: \"2aad88e5-b35d-4876-8a3b-c8b0b64f8570\") " pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:40:05.461210 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:05.461133    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:40:05.461210 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:05.461147    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:40:05.461210 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:05.461156    2578 projected.go:196] Error preparing data for projected volume kube-api-access-lpgjd for pod openshift-network-diagnostics/network-check-target-wbfzv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:40:05.461210 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:05.461212    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd podName:2aad88e5-b35d-4876-8a3b-c8b0b64f8570 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:37.461198635 +0000 UTC m=+65.248762768 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lpgjd" (UniqueName: "kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd") pod "network-check-target-wbfzv" (UID: "2aad88e5-b35d-4876-8a3b-c8b0b64f8570") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:40:05.703094 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:05.702900    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:40:05.703094 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:05.702916    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:40:05.703094 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:05.702905    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:40:05.703094 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:05.703024    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wbfzv" podUID="2aad88e5-b35d-4876-8a3b-c8b0b64f8570"
Sep 16 18:40:05.703309 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:05.703133    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:40:05.703309 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:05.703221    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c8l7h" podUID="73a1706e-dce9-475e-b553-642999cc42c0"
Sep 16 18:40:06.054622 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.054541    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-234.ec2.internal" event="NodeReady"
Sep 16 18:40:06.054992 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.054679    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 18:40:06.130691 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.130660    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-h6s5l"]
Sep 16 18:40:06.147722 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.147690    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-crt2q"]
Sep 16 18:40:06.147903 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.147817    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-h6s5l"
Sep 16 18:40:06.150564 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.150537    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 18:40:06.150721 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.150692    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 18:40:06.150823 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.150805    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 18:40:06.151244 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.151216    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7zs8w\""
Sep 16 18:40:06.160166 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.160144    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:06.161917 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.161884    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-h6s5l"]
Sep 16 18:40:06.162690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.162665    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-crt2q"]
Sep 16 18:40:06.162801 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.162791    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 18:40:06.162859 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.162800    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-24cbm\""
Sep 16 18:40:06.163066 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.163047    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 18:40:06.267779 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.267744    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-tmp-dir\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:06.267779 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.267792    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-config-volume\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:06.268039 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.267906    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mc2t\" (UniqueName: \"kubernetes.io/projected/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-kube-api-access-2mc2t\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:06.268039 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.267944    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:06.268039 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.268001    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f21dc980-1bae-414a-be6b-7145bcc7c7f5-cert\") pod \"ingress-canary-h6s5l\" (UID: \"f21dc980-1bae-414a-be6b-7145bcc7c7f5\") " pod="openshift-ingress-canary/ingress-canary-h6s5l"
Sep 16 18:40:06.268039 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.268028    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtkkh\" (UniqueName: \"kubernetes.io/projected/f21dc980-1bae-414a-be6b-7145bcc7c7f5-kube-api-access-qtkkh\") pod \"ingress-canary-h6s5l\" (UID: \"f21dc980-1bae-414a-be6b-7145bcc7c7f5\") " pod="openshift-ingress-canary/ingress-canary-h6s5l"
Sep 16 18:40:06.369235 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.369199    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-tmp-dir\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:06.369235 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.369247    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-config-volume\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:06.369510 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.369400    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mc2t\" (UniqueName: \"kubernetes.io/projected/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-kube-api-access-2mc2t\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:06.369510 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.369439    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:06.369510 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.369484    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f21dc980-1bae-414a-be6b-7145bcc7c7f5-cert\") pod \"ingress-canary-h6s5l\" (UID: \"f21dc980-1bae-414a-be6b-7145bcc7c7f5\") " pod="openshift-ingress-canary/ingress-canary-h6s5l"
Sep 16 18:40:06.369657 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.369517    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtkkh\" (UniqueName: \"kubernetes.io/projected/f21dc980-1bae-414a-be6b-7145bcc7c7f5-kube-api-access-qtkkh\") pod \"ingress-canary-h6s5l\" (UID: \"f21dc980-1bae-414a-be6b-7145bcc7c7f5\") " pod="openshift-ingress-canary/ingress-canary-h6s5l"
Sep 16 18:40:06.369709 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.369689    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-tmp-dir\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:06.369790 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:06.369773    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:40:06.369846 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:06.369835    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls podName:6fbdb2ba-c71f-47e6-bb39-1f2aa806df61 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:06.869817877 +0000 UTC m=+34.657382023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls") pod "dns-default-crt2q" (UID: "6fbdb2ba-c71f-47e6-bb39-1f2aa806df61") : secret "dns-default-metrics-tls" not found
Sep 16 18:40:06.369930 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.369912    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-config-volume\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:06.374311 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.374286    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f21dc980-1bae-414a-be6b-7145bcc7c7f5-cert\") pod \"ingress-canary-h6s5l\" (UID: \"f21dc980-1bae-414a-be6b-7145bcc7c7f5\") " pod="openshift-ingress-canary/ingress-canary-h6s5l"
Sep 16 18:40:06.393341 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.393314    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mc2t\" (UniqueName: \"kubernetes.io/projected/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-kube-api-access-2mc2t\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:06.393503 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.393470    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtkkh\" (UniqueName: \"kubernetes.io/projected/f21dc980-1bae-414a-be6b-7145bcc7c7f5-kube-api-access-qtkkh\") pod \"ingress-canary-h6s5l\" (UID: \"f21dc980-1bae-414a-be6b-7145bcc7c7f5\") " pod="openshift-ingress-canary/ingress-canary-h6s5l"
Sep 16 18:40:06.458310 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.458275    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-h6s5l"
Sep 16 18:40:06.777551 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.777271    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-h6s5l"]
Sep 16 18:40:06.781280 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:40:06.781249    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf21dc980_1bae_414a_be6b_7145bcc7c7f5.slice/crio-2206eb915a9525046bdeb0c8dd78ecc726d94424435cbd76b9696970f51a31de WatchSource:0}: Error finding container 2206eb915a9525046bdeb0c8dd78ecc726d94424435cbd76b9696970f51a31de: Status 404 returned error can't find the container with id 2206eb915a9525046bdeb0c8dd78ecc726d94424435cbd76b9696970f51a31de
Sep 16 18:40:06.874213 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.874158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:06.874493 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:06.874318    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:40:06.874493 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:06.874396    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls podName:6fbdb2ba-c71f-47e6-bb39-1f2aa806df61 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:07.874373746 +0000 UTC m=+35.661937883 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls") pod "dns-default-crt2q" (UID: "6fbdb2ba-c71f-47e6-bb39-1f2aa806df61") : secret "dns-default-metrics-tls" not found
Sep 16 18:40:06.984633 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:06.984596    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-h6s5l" event={"ID":"f21dc980-1bae-414a-be6b-7145bcc7c7f5","Type":"ContainerStarted","Data":"2206eb915a9525046bdeb0c8dd78ecc726d94424435cbd76b9696970f51a31de"}
Sep 16 18:40:07.703538 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:07.703495    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:40:07.704003 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:07.703518    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:40:07.704003 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:07.703639    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:40:07.708767 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:07.708736    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 18:40:07.709204 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:07.709157    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 18:40:07.710054 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:07.709703    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rmdqb\""
Sep 16 18:40:07.710054 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:07.709804    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 18:40:07.710054 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:07.709920    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l6kxj\""
Sep 16 18:40:07.710256 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:07.710224    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 18:40:07.885224 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:07.885185    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:07.885413 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:07.885333    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:40:07.885413 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:07.885399    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls podName:6fbdb2ba-c71f-47e6-bb39-1f2aa806df61 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:09.885380218 +0000 UTC m=+37.672944365 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls") pod "dns-default-crt2q" (UID: "6fbdb2ba-c71f-47e6-bb39-1f2aa806df61") : secret "dns-default-metrics-tls" not found
Sep 16 18:40:09.901712 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:09.901671    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:09.902288 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:09.901854    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:40:09.902288 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:09.901936    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls podName:6fbdb2ba-c71f-47e6-bb39-1f2aa806df61 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:13.90191411 +0000 UTC m=+41.689478257 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls") pod "dns-default-crt2q" (UID: "6fbdb2ba-c71f-47e6-bb39-1f2aa806df61") : secret "dns-default-metrics-tls" not found
Sep 16 18:40:10.997128 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:10.997094    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-h6s5l" event={"ID":"f21dc980-1bae-414a-be6b-7145bcc7c7f5","Type":"ContainerStarted","Data":"c87397aa152bac638cbe34d61154b78def0cd741bb57118c6454c9176d3fa481"}
Sep 16 18:40:10.999890 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:10.999869    2578 generic.go:356] "Generic (PLEG): container finished" podID="1df53334-7b61-47d0-9787-1dc604c84896" containerID="f7f2f8395760c43847ed4edbd17534fef987fb778728bef444d84cc3d606499e" exitCode=0
Sep 16 18:40:10.999989 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:10.999907    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nsrzq" event={"ID":"1df53334-7b61-47d0-9787-1dc604c84896","Type":"ContainerDied","Data":"f7f2f8395760c43847ed4edbd17534fef987fb778728bef444d84cc3d606499e"}
Sep 16 18:40:11.015791 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:11.015743    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-h6s5l" podStartSLOduration=1.740673575 podStartE2EDuration="5.015728747s" totalImagesPullingTime="3.275055172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:11.014362413 +0000 UTC m=+38.801926589" watchObservedRunningTime="2026-09-16 18:40:11.015728747 +0000 UTC m=+38.803292935"
Sep 16 18:40:12.005101 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:12.005063    2578 generic.go:356] "Generic (PLEG): container finished" podID="1df53334-7b61-47d0-9787-1dc604c84896" containerID="76c90bdd13ab714bd241410ba571f4b683bac1307abd58b33bcb76d41925057e" exitCode=0
Sep 16 18:40:12.005525 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:12.005146    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nsrzq" event={"ID":"1df53334-7b61-47d0-9787-1dc604c84896","Type":"ContainerDied","Data":"76c90bdd13ab714bd241410ba571f4b683bac1307abd58b33bcb76d41925057e"}
Sep 16 18:40:13.010779 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:13.010743    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nsrzq" event={"ID":"1df53334-7b61-47d0-9787-1dc604c84896","Type":"ContainerStarted","Data":"32f1fe658ec9e746f96954b45bdb396421dcea5caea7d117070cd55268ff1d27"}
Sep 16 18:40:13.036569 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:13.036515    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nsrzq" podStartSLOduration=7.084448076 podStartE2EDuration="40.036498991s" totalImagesPullingTime="32.952050915s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:39:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:13.035509308 +0000 UTC m=+40.823073462" watchObservedRunningTime="2026-09-16 18:40:13.036498991 +0000 UTC m=+40.824063146"
Sep 16 18:40:13.930016 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:13.929980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:13.930226 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:13.930099    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:40:13.930226 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:13.930152    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls podName:6fbdb2ba-c71f-47e6-bb39-1f2aa806df61 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:21.930138268 +0000 UTC m=+49.717702401 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls") pod "dns-default-crt2q" (UID: "6fbdb2ba-c71f-47e6-bb39-1f2aa806df61") : secret "dns-default-metrics-tls" not found
Sep 16 18:40:21.984157 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:21.984111    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:21.984623 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:21.984300    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:40:21.984623 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:21.984385    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls podName:6fbdb2ba-c71f-47e6-bb39-1f2aa806df61 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:37.984364083 +0000 UTC m=+65.771928239 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls") pod "dns-default-crt2q" (UID: "6fbdb2ba-c71f-47e6-bb39-1f2aa806df61") : secret "dns-default-metrics-tls" not found
Sep 16 18:40:32.978683 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:32.978656    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gbxkd"
Sep 16 18:40:37.389413 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:37.389377    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs\") pod \"network-metrics-daemon-282m9\" (UID: \"87684818-77c6-45bc-bee8-61a206348549\") " pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:40:37.389788 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:37.389422    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret\") pod \"global-pull-secret-syncer-c8l7h\" (UID: \"73a1706e-dce9-475e-b553-642999cc42c0\") " pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:40:37.391979 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:37.391959    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 18:40:37.392030 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:37.391963    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 18:40:37.399890 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:37.399868    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 18:40:37.399958 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:37.399947    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs podName:87684818-77c6-45bc-bee8-61a206348549 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:41.399926695 +0000 UTC m=+129.187490831 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs") pod "network-metrics-daemon-282m9" (UID: "87684818-77c6-45bc-bee8-61a206348549") : secret "metrics-daemon-secret" not found
Sep 16 18:40:37.402795 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:37.402764    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/73a1706e-dce9-475e-b553-642999cc42c0-original-pull-secret\") pod \"global-pull-secret-syncer-c8l7h\" (UID: \"73a1706e-dce9-475e-b553-642999cc42c0\") " pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:40:37.433257 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:37.433223    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c8l7h"
Sep 16 18:40:37.490736 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:37.490698    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpgjd\" (UniqueName: \"kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd\") pod \"network-check-target-wbfzv\" (UID: \"2aad88e5-b35d-4876-8a3b-c8b0b64f8570\") " pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:40:37.493587 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:37.493562    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 18:40:37.503972 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:37.503946    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 18:40:37.514741 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:37.514716    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpgjd\" (UniqueName: \"kubernetes.io/projected/2aad88e5-b35d-4876-8a3b-c8b0b64f8570-kube-api-access-lpgjd\") pod \"network-check-target-wbfzv\" (UID: \"2aad88e5-b35d-4876-8a3b-c8b0b64f8570\") " pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:40:37.703999 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:37.703929    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c8l7h"]
Sep 16 18:40:37.707201 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:40:37.707159    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a1706e_dce9_475e_b553_642999cc42c0.slice/crio-a79a11f1e50ddcd5943171b27597e26ac5c958c79566e35eb5f2868202042d9d WatchSource:0}: Error finding container a79a11f1e50ddcd5943171b27597e26ac5c958c79566e35eb5f2868202042d9d: Status 404 returned error can't find the container with id a79a11f1e50ddcd5943171b27597e26ac5c958c79566e35eb5f2868202042d9d
Sep 16 18:40:37.719272 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:37.719243    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rmdqb\""
Sep 16 18:40:37.727386 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:37.727360    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:40:37.996150 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:37.996061    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:40:37.996304 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:37.996235    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:40:37.996344 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:40:37.996314    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls podName:6fbdb2ba-c71f-47e6-bb39-1f2aa806df61 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:09.996297948 +0000 UTC m=+97.783862086 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls") pod "dns-default-crt2q" (UID: "6fbdb2ba-c71f-47e6-bb39-1f2aa806df61") : secret "dns-default-metrics-tls" not found
Sep 16 18:40:38.001340 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:38.001305    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wbfzv"]
Sep 16 18:40:38.004735 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:40:38.004713    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aad88e5_b35d_4876_8a3b_c8b0b64f8570.slice/crio-4b36a4bee52673572f7035e95aa96b2b1effce76c61d41437c3f1ff23c2e29ff WatchSource:0}: Error finding container 4b36a4bee52673572f7035e95aa96b2b1effce76c61d41437c3f1ff23c2e29ff: Status 404 returned error can't find the container with id 4b36a4bee52673572f7035e95aa96b2b1effce76c61d41437c3f1ff23c2e29ff
Sep 16 18:40:38.068954 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:38.068915    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c8l7h" event={"ID":"73a1706e-dce9-475e-b553-642999cc42c0","Type":"ContainerStarted","Data":"a79a11f1e50ddcd5943171b27597e26ac5c958c79566e35eb5f2868202042d9d"}
Sep 16 18:40:38.069956 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:38.069934    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wbfzv" event={"ID":"2aad88e5-b35d-4876-8a3b-c8b0b64f8570","Type":"ContainerStarted","Data":"4b36a4bee52673572f7035e95aa96b2b1effce76c61d41437c3f1ff23c2e29ff"}
Sep 16 18:40:43.083882 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:43.083839    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wbfzv" event={"ID":"2aad88e5-b35d-4876-8a3b-c8b0b64f8570","Type":"ContainerStarted","Data":"eba108f5d358ba3f7313f16e23308b67940defdb1289ef3b93f2f066388e9f63"}
Sep 16 18:40:43.084387 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:43.083941    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:40:43.085372 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:43.085350    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c8l7h" event={"ID":"73a1706e-dce9-475e-b553-642999cc42c0","Type":"ContainerStarted","Data":"6779d89be28ee32aa17481c675d8fb23bf31c9b487a9d238c17f8fbe34b786c9"}
Sep 16 18:40:43.117401 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:43.117355    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wbfzv" podStartSLOduration=65.660922113 podStartE2EDuration="1m10.117338279s" totalImagesPullingTime="4.456416166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:43.113668878 +0000 UTC m=+70.901233034" watchObservedRunningTime="2026-09-16 18:40:43.117338279 +0000 UTC m=+70.904902416"
Sep 16 18:40:43.144631 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:40:43.144576    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-c8l7h" podStartSLOduration=65.388055818 podStartE2EDuration="1m10.144559795s" totalImagesPullingTime="4.756503977s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:43.144429653 +0000 UTC m=+70.931993807" watchObservedRunningTime="2026-09-16 18:40:43.144559795 +0000 UTC m=+70.932123951"
Sep 16 18:41:09.999129 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:09.999087    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:41:09.999570 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:09.999231    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:41:09.999570 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:09.999301    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls podName:6fbdb2ba-c71f-47e6-bb39-1f2aa806df61 nodeName:}" failed. No retries permitted until 2026-09-16 18:42:13.999282059 +0000 UTC m=+161.786846193 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls") pod "dns-default-crt2q" (UID: "6fbdb2ba-c71f-47e6-bb39-1f2aa806df61") : secret "dns-default-metrics-tls" not found
Sep 16 18:41:14.089904 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:14.089869    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wbfzv"
Sep 16 18:41:31.960243 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:31.960203    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-vhrf8"]
Sep 16 18:41:31.962447 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:31.962429    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:31.970108 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:31.970051    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"service-ca-bundle\" is forbidden: User \"system:node:ip-10-0-134-234.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-134-234.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" type="*v1.ConfigMap"
Sep 16 18:41:31.970108 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:31.970089    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"openshift-insights-serving-cert\" is forbidden: User \"system:node:ip-10-0-134-234.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-134-234.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" type="*v1.Secret"
Sep 16 18:41:31.970296 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:31.970097    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"insights-operator-5844b856db-vhrf8\" is forbidden: User \"system:node:ip-10-0-134-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-134-234.ec2.internal' and this object" podUID="306e53f3-37de-49c6-a491-f9faf67d7f2a" pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:31.970296 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:31.970102    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-134-234.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-134-234.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Sep 16 18:41:31.970296 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:31.970143    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"trusted-ca-bundle\" is forbidden: User \"system:node:ip-10-0-134-234.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-134-234.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" type="*v1.ConfigMap"
Sep 16 18:41:31.970296 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:31.970154    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"operator-dockercfg-f6vjn\" is forbidden: User \"system:node:ip-10-0-134-234.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-134-234.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"operator-dockercfg-f6vjn\"" type="*v1.Secret"
Sep 16 18:41:31.970296 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:31.970229    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-134-234.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-134-234.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Sep 16 18:41:32.000518 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.000487    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9d6k"]
Sep 16 18:41:32.002389 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.002374    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9d6k"
Sep 16 18:41:32.009264 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.009242    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 16 18:41:32.009851 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.009834    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-d6c29\""
Sep 16 18:41:32.014576 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.014559    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:41:32.026644 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.026622    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9d6k"]
Sep 16 18:41:32.049371 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.049338    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/306e53f3-37de-49c6-a491-f9faf67d7f2a-service-ca-bundle\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:32.049511 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.049377    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/306e53f3-37de-49c6-a491-f9faf67d7f2a-tmp\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:32.049511 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.049395    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/306e53f3-37de-49c6-a491-f9faf67d7f2a-snapshots\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:32.049511 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.049416    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd4r2\" (UniqueName: \"kubernetes.io/projected/6a1e4944-7d0e-4a4c-8cd0-b1a762f4798c-kube-api-access-kd4r2\") pod \"volume-data-source-validator-86695b7cc7-b9d6k\" (UID: \"6a1e4944-7d0e-4a4c-8cd0-b1a762f4798c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9d6k"
Sep 16 18:41:32.049511 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.049484    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/306e53f3-37de-49c6-a491-f9faf67d7f2a-serving-cert\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:32.049511 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.049501    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/306e53f3-37de-49c6-a491-f9faf67d7f2a-trusted-ca-bundle\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:32.049692 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.049547    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvjtm\" (UniqueName: \"kubernetes.io/projected/306e53f3-37de-49c6-a491-f9faf67d7f2a-kube-api-access-qvjtm\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:32.079705 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.079670    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-vhrf8"]
Sep 16 18:41:32.096818 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.096790    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"]
Sep 16 18:41:32.098796 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.098780    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:41:32.110249 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.110227    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-gwwbr"]
Sep 16 18:41:32.111005 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.110980    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 16 18:41:32.111005 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.110999    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 18:41:32.111149 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.111006    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 16 18:41:32.111328 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.111287    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 18:41:32.111328 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.111289    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mz8qf\""
Sep 16 18:41:32.112228 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.112213    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-gwwbr"
Sep 16 18:41:32.125574 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.125554    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-l828q\""
Sep 16 18:41:32.149973 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.149945    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvjtm\" (UniqueName: \"kubernetes.io/projected/306e53f3-37de-49c6-a491-f9faf67d7f2a-kube-api-access-qvjtm\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:32.150092 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.149980    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxcgv\" (UniqueName: \"kubernetes.io/projected/94e8727d-8666-4010-8a30-0307100e0825-kube-api-access-sxcgv\") pod \"network-check-source-6f8b768455-gwwbr\" (UID: \"94e8727d-8666-4010-8a30-0307100e0825\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-gwwbr"
Sep 16 18:41:32.150092 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.150017    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/306e53f3-37de-49c6-a491-f9faf67d7f2a-service-ca-bundle\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:32.150092 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.150040    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tffz\" (UniqueName: \"kubernetes.io/projected/54176761-3f77-4f63-ada8-c65ec07f450c-kube-api-access-4tffz\") pod \"cluster-monitoring-operator-6cd954b897-4vqx5\" (UID: \"54176761-3f77-4f63-ada8-c65ec07f450c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:41:32.150092 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.150068    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/306e53f3-37de-49c6-a491-f9faf67d7f2a-tmp\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:32.150458 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.150090    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4vqx5\" (UID: \"54176761-3f77-4f63-ada8-c65ec07f450c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:41:32.150458 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.150142    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/306e53f3-37de-49c6-a491-f9faf67d7f2a-snapshots\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:32.150458 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.150194    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd4r2\" (UniqueName: \"kubernetes.io/projected/6a1e4944-7d0e-4a4c-8cd0-b1a762f4798c-kube-api-access-kd4r2\") pod \"volume-data-source-validator-86695b7cc7-b9d6k\" (UID: \"6a1e4944-7d0e-4a4c-8cd0-b1a762f4798c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9d6k"
Sep 16 18:41:32.150458 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.150224    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/54176761-3f77-4f63-ada8-c65ec07f450c-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-4vqx5\" (UID: \"54176761-3f77-4f63-ada8-c65ec07f450c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:41:32.150458 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.150261    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/306e53f3-37de-49c6-a491-f9faf67d7f2a-serving-cert\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:32.150458 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.150290    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/306e53f3-37de-49c6-a491-f9faf67d7f2a-trusted-ca-bundle\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:32.150458 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.150407    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/306e53f3-37de-49c6-a491-f9faf67d7f2a-tmp\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:32.150770 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.150717    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/306e53f3-37de-49c6-a491-f9faf67d7f2a-snapshots\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:32.168836 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.168812    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"]
Sep 16 18:41:32.169457 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.169433    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-gwwbr"]
Sep 16 18:41:32.182946 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.182916    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd4r2\" (UniqueName: \"kubernetes.io/projected/6a1e4944-7d0e-4a4c-8cd0-b1a762f4798c-kube-api-access-kd4r2\") pod \"volume-data-source-validator-86695b7cc7-b9d6k\" (UID: \"6a1e4944-7d0e-4a4c-8cd0-b1a762f4798c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9d6k"
Sep 16 18:41:32.250775 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.250688    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tffz\" (UniqueName: \"kubernetes.io/projected/54176761-3f77-4f63-ada8-c65ec07f450c-kube-api-access-4tffz\") pod \"cluster-monitoring-operator-6cd954b897-4vqx5\" (UID: \"54176761-3f77-4f63-ada8-c65ec07f450c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:41:32.250775 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.250725    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4vqx5\" (UID: \"54176761-3f77-4f63-ada8-c65ec07f450c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:41:32.250967 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.250778    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/54176761-3f77-4f63-ada8-c65ec07f450c-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-4vqx5\" (UID: \"54176761-3f77-4f63-ada8-c65ec07f450c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:41:32.250967 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.250838    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxcgv\" (UniqueName: \"kubernetes.io/projected/94e8727d-8666-4010-8a30-0307100e0825-kube-api-access-sxcgv\") pod \"network-check-source-6f8b768455-gwwbr\" (UID: \"94e8727d-8666-4010-8a30-0307100e0825\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-gwwbr"
Sep 16 18:41:32.250967 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:32.250889    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 18:41:32.250967 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:32.250957    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls podName:54176761-3f77-4f63-ada8-c65ec07f450c nodeName:}" failed. No retries permitted until 2026-09-16 18:41:32.750941612 +0000 UTC m=+120.538505750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4vqx5" (UID: "54176761-3f77-4f63-ada8-c65ec07f450c") : secret "cluster-monitoring-operator-tls" not found
Sep 16 18:41:32.251485 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.251467    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/54176761-3f77-4f63-ada8-c65ec07f450c-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-4vqx5\" (UID: \"54176761-3f77-4f63-ada8-c65ec07f450c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:41:32.257242 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.257222    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z"]
Sep 16 18:41:32.259516 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.259484    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-ttc6f"]
Sep 16 18:41:32.259655 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.259619    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z"
Sep 16 18:41:32.261716 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.261699    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:41:32.271055 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.271030    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-cw4w5\""
Sep 16 18:41:32.275146 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.275127    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:41:32.277774 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.277755    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 16 18:41:32.277887 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.277872    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 16 18:41:32.284363 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.284346    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 16 18:41:32.284558 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.284541    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:41:32.284632 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.284573    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 16 18:41:32.299963 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.299946    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 16 18:41:32.310343 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.310328    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9d6k"
Sep 16 18:41:32.313516 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.313493    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tffz\" (UniqueName: \"kubernetes.io/projected/54176761-3f77-4f63-ada8-c65ec07f450c-kube-api-access-4tffz\") pod \"cluster-monitoring-operator-6cd954b897-4vqx5\" (UID: \"54176761-3f77-4f63-ada8-c65ec07f450c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:41:32.326200 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.326157    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pgdkz\""
Sep 16 18:41:32.327610 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.327586    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z"]
Sep 16 18:41:32.329313 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.329285    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxcgv\" (UniqueName: \"kubernetes.io/projected/94e8727d-8666-4010-8a30-0307100e0825-kube-api-access-sxcgv\") pod \"network-check-source-6f8b768455-gwwbr\" (UID: \"94e8727d-8666-4010-8a30-0307100e0825\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-gwwbr"
Sep 16 18:41:32.329713 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.329694    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-ttc6f"]
Sep 16 18:41:32.352105 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.352072    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-t6m7z\" (UID: \"19ab637e-2e0e-4e42-9a29-4da465a11b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z"
Sep 16 18:41:32.352270 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.352122    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/845d9b0a-5790-4a60-894b-f6b9adce026d-config\") pod \"console-operator-696884fb9c-ttc6f\" (UID: \"845d9b0a-5790-4a60-894b-f6b9adce026d\") " pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:41:32.352270 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.352157    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/845d9b0a-5790-4a60-894b-f6b9adce026d-trusted-ca\") pod \"console-operator-696884fb9c-ttc6f\" (UID: \"845d9b0a-5790-4a60-894b-f6b9adce026d\") " pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:41:32.352270 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.352187    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfppp\" (UniqueName: \"kubernetes.io/projected/19ab637e-2e0e-4e42-9a29-4da465a11b5e-kube-api-access-gfppp\") pod \"cluster-samples-operator-86499c89c4-t6m7z\" (UID: \"19ab637e-2e0e-4e42-9a29-4da465a11b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z"
Sep 16 18:41:32.352270 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.352214    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/845d9b0a-5790-4a60-894b-f6b9adce026d-serving-cert\") pod \"console-operator-696884fb9c-ttc6f\" (UID: \"845d9b0a-5790-4a60-894b-f6b9adce026d\") " pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:41:32.352270 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.352231    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhpd9\" (UniqueName: \"kubernetes.io/projected/845d9b0a-5790-4a60-894b-f6b9adce026d-kube-api-access-zhpd9\") pod \"console-operator-696884fb9c-ttc6f\" (UID: \"845d9b0a-5790-4a60-894b-f6b9adce026d\") " pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:41:32.366424 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.366397    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 16 18:41:32.390018 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.389990    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"]
Sep 16 18:41:32.392554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.392538    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"
Sep 16 18:41:32.404579 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.404560    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 16 18:41:32.405097 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.405079    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gdrdg\""
Sep 16 18:41:32.409046 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.409028    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"]
Sep 16 18:41:32.410142 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.410125    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 16 18:41:32.419966 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.419944    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-gwwbr"
Sep 16 18:41:32.453191 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.453143    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-vr97q\" (UID: \"6d1b4b46-d618-4ceb-8e69-d034dce9b435\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"
Sep 16 18:41:32.453344 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.453246    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/845d9b0a-5790-4a60-894b-f6b9adce026d-trusted-ca\") pod \"console-operator-696884fb9c-ttc6f\" (UID: \"845d9b0a-5790-4a60-894b-f6b9adce026d\") " pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:41:32.453344 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.453277    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfppp\" (UniqueName: \"kubernetes.io/projected/19ab637e-2e0e-4e42-9a29-4da465a11b5e-kube-api-access-gfppp\") pod \"cluster-samples-operator-86499c89c4-t6m7z\" (UID: \"19ab637e-2e0e-4e42-9a29-4da465a11b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z"
Sep 16 18:41:32.453344 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.453326    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/845d9b0a-5790-4a60-894b-f6b9adce026d-serving-cert\") pod \"console-operator-696884fb9c-ttc6f\" (UID: \"845d9b0a-5790-4a60-894b-f6b9adce026d\") " pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:41:32.453519 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.453353    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhpd9\" (UniqueName: \"kubernetes.io/projected/845d9b0a-5790-4a60-894b-f6b9adce026d-kube-api-access-zhpd9\") pod \"console-operator-696884fb9c-ttc6f\" (UID: \"845d9b0a-5790-4a60-894b-f6b9adce026d\") " pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:41:32.453519 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.453428    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-t6m7z\" (UID: \"19ab637e-2e0e-4e42-9a29-4da465a11b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z"
Sep 16 18:41:32.453519 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.453455    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6d1b4b46-d618-4ceb-8e69-d034dce9b435-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-vr97q\" (UID: \"6d1b4b46-d618-4ceb-8e69-d034dce9b435\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"
Sep 16 18:41:32.453519 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.453501    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/845d9b0a-5790-4a60-894b-f6b9adce026d-config\") pod \"console-operator-696884fb9c-ttc6f\" (UID: \"845d9b0a-5790-4a60-894b-f6b9adce026d\") " pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:41:32.454450 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:32.454136    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 18:41:32.454450 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.454216    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/845d9b0a-5790-4a60-894b-f6b9adce026d-config\") pod \"console-operator-696884fb9c-ttc6f\" (UID: \"845d9b0a-5790-4a60-894b-f6b9adce026d\") " pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:41:32.454450 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:32.454242    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls podName:19ab637e-2e0e-4e42-9a29-4da465a11b5e nodeName:}" failed. No retries permitted until 2026-09-16 18:41:32.954222568 +0000 UTC m=+120.741786705 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-t6m7z" (UID: "19ab637e-2e0e-4e42-9a29-4da465a11b5e") : secret "samples-operator-tls" not found
Sep 16 18:41:32.454450 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.454322    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/845d9b0a-5790-4a60-894b-f6b9adce026d-trusted-ca\") pod \"console-operator-696884fb9c-ttc6f\" (UID: \"845d9b0a-5790-4a60-894b-f6b9adce026d\") " pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:41:32.456606 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.456582    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/845d9b0a-5790-4a60-894b-f6b9adce026d-serving-cert\") pod \"console-operator-696884fb9c-ttc6f\" (UID: \"845d9b0a-5790-4a60-894b-f6b9adce026d\") " pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:41:32.464796 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.464776    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfppp\" (UniqueName: \"kubernetes.io/projected/19ab637e-2e0e-4e42-9a29-4da465a11b5e-kube-api-access-gfppp\") pod \"cluster-samples-operator-86499c89c4-t6m7z\" (UID: \"19ab637e-2e0e-4e42-9a29-4da465a11b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z"
Sep 16 18:41:32.476587 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.476563    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhpd9\" (UniqueName: \"kubernetes.io/projected/845d9b0a-5790-4a60-894b-f6b9adce026d-kube-api-access-zhpd9\") pod \"console-operator-696884fb9c-ttc6f\" (UID: \"845d9b0a-5790-4a60-894b-f6b9adce026d\") " pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:41:32.554030 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.553935    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6d1b4b46-d618-4ceb-8e69-d034dce9b435-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-vr97q\" (UID: \"6d1b4b46-d618-4ceb-8e69-d034dce9b435\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"
Sep 16 18:41:32.554030 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.553982    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-vr97q\" (UID: \"6d1b4b46-d618-4ceb-8e69-d034dce9b435\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"
Sep 16 18:41:32.554230 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:32.554079    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 18:41:32.554230 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:32.554145    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert podName:6d1b4b46-d618-4ceb-8e69-d034dce9b435 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:33.054131816 +0000 UTC m=+120.841695949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-vr97q" (UID: "6d1b4b46-d618-4ceb-8e69-d034dce9b435") : secret "networking-console-plugin-cert" not found
Sep 16 18:41:32.554627 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.554605    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6d1b4b46-d618-4ceb-8e69-d034dce9b435-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-vr97q\" (UID: \"6d1b4b46-d618-4ceb-8e69-d034dce9b435\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"
Sep 16 18:41:32.574091 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.574060    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:41:32.592956 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.592929    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9d6k"]
Sep 16 18:41:32.595949 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:41:32.595922    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a1e4944_7d0e_4a4c_8cd0_b1a762f4798c.slice/crio-807802e2b6d4113a8ac853c7215b2eb8e7c2c7d0722efadaa5e19294eea41607 WatchSource:0}: Error finding container 807802e2b6d4113a8ac853c7215b2eb8e7c2c7d0722efadaa5e19294eea41607: Status 404 returned error can't find the container with id 807802e2b6d4113a8ac853c7215b2eb8e7c2c7d0722efadaa5e19294eea41607
Sep 16 18:41:32.736669 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.736637    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-gwwbr"]
Sep 16 18:41:32.739655 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:41:32.739633    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94e8727d_8666_4010_8a30_0307100e0825.slice/crio-c28ac024d1763d1cd4517f0c43f93640ac5f06049f603c0520c6e605fea351e1 WatchSource:0}: Error finding container c28ac024d1763d1cd4517f0c43f93640ac5f06049f603c0520c6e605fea351e1: Status 404 returned error can't find the container with id c28ac024d1763d1cd4517f0c43f93640ac5f06049f603c0520c6e605fea351e1
Sep 16 18:41:32.754726 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.754701    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4vqx5\" (UID: \"54176761-3f77-4f63-ada8-c65ec07f450c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:41:32.754865 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:32.754845    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 18:41:32.754934 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:32.754922    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls podName:54176761-3f77-4f63-ada8-c65ec07f450c nodeName:}" failed. No retries permitted until 2026-09-16 18:41:33.754902671 +0000 UTC m=+121.542466823 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4vqx5" (UID: "54176761-3f77-4f63-ada8-c65ec07f450c") : secret "cluster-monitoring-operator-tls" not found
Sep 16 18:41:32.857153 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.857135    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-f6vjn\""
Sep 16 18:41:32.867624 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.867594    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-ttc6f"]
Sep 16 18:41:32.871298 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:41:32.871273    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod845d9b0a_5790_4a60_894b_f6b9adce026d.slice/crio-dbd6c1bf9c159cc29d1fbb275617a0d13478ba29d380c60942b36e06faf90cbd WatchSource:0}: Error finding container dbd6c1bf9c159cc29d1fbb275617a0d13478ba29d380c60942b36e06faf90cbd: Status 404 returned error can't find the container with id dbd6c1bf9c159cc29d1fbb275617a0d13478ba29d380c60942b36e06faf90cbd
Sep 16 18:41:32.956089 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:32.956052    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-t6m7z\" (UID: \"19ab637e-2e0e-4e42-9a29-4da465a11b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z"
Sep 16 18:41:32.956290 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:32.956245    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 18:41:32.956349 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:32.956340    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls podName:19ab637e-2e0e-4e42-9a29-4da465a11b5e nodeName:}" failed. No retries permitted until 2026-09-16 18:41:33.956319093 +0000 UTC m=+121.743883228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-t6m7z" (UID: "19ab637e-2e0e-4e42-9a29-4da465a11b5e") : secret "samples-operator-tls" not found
Sep 16 18:41:33.016322 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.016283    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 16 18:41:33.021761 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.021734    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/306e53f3-37de-49c6-a491-f9faf67d7f2a-trusted-ca-bundle\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:33.056486 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.056447    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-vr97q\" (UID: \"6d1b4b46-d618-4ceb-8e69-d034dce9b435\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"
Sep 16 18:41:33.056667 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:33.056636    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 18:41:33.056731 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:33.056722    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert podName:6d1b4b46-d618-4ceb-8e69-d034dce9b435 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:34.056698157 +0000 UTC m=+121.844262294 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-vr97q" (UID: "6d1b4b46-d618-4ceb-8e69-d034dce9b435") : secret "networking-console-plugin-cert" not found
Sep 16 18:41:33.151072 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:33.150971    2578 secret.go:189] Couldn't get secret openshift-insights/openshift-insights-serving-cert: failed to sync secret cache: timed out waiting for the condition
Sep 16 18:41:33.151072 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:33.151067    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306e53f3-37de-49c6-a491-f9faf67d7f2a-serving-cert podName:306e53f3-37de-49c6-a491-f9faf67d7f2a nodeName:}" failed. No retries permitted until 2026-09-16 18:41:33.651044755 +0000 UTC m=+121.438608907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "serving-cert" (UniqueName: "kubernetes.io/secret/306e53f3-37de-49c6-a491-f9faf67d7f2a-serving-cert") pod "insights-operator-5844b856db-vhrf8" (UID: "306e53f3-37de-49c6-a491-f9faf67d7f2a") : failed to sync secret cache: timed out waiting for the condition
Sep 16 18:41:33.151351 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:33.150971    2578 configmap.go:193] Couldn't get configMap openshift-insights/service-ca-bundle: failed to sync configmap cache: timed out waiting for the condition
Sep 16 18:41:33.151351 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:33.151162    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/306e53f3-37de-49c6-a491-f9faf67d7f2a-service-ca-bundle podName:306e53f3-37de-49c6-a491-f9faf67d7f2a nodeName:}" failed. No retries permitted until 2026-09-16 18:41:33.651141992 +0000 UTC m=+121.438706153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/306e53f3-37de-49c6-a491-f9faf67d7f2a-service-ca-bundle") pod "insights-operator-5844b856db-vhrf8" (UID: "306e53f3-37de-49c6-a491-f9faf67d7f2a") : failed to sync configmap cache: timed out waiting for the condition
Sep 16 18:41:33.172327 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:33.172297    2578 projected.go:291] Couldn't get configMap openshift-insights/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Sep 16 18:41:33.210872 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.210819    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-gwwbr" event={"ID":"94e8727d-8666-4010-8a30-0307100e0825","Type":"ContainerStarted","Data":"c34b8068879942205c8e11852358c7ea60d97c03d6242612b119b6c8cdda8076"}
Sep 16 18:41:33.211048 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.210918    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-gwwbr" event={"ID":"94e8727d-8666-4010-8a30-0307100e0825","Type":"ContainerStarted","Data":"c28ac024d1763d1cd4517f0c43f93640ac5f06049f603c0520c6e605fea351e1"}
Sep 16 18:41:33.212136 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.212091    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9d6k" event={"ID":"6a1e4944-7d0e-4a4c-8cd0-b1a762f4798c","Type":"ContainerStarted","Data":"807802e2b6d4113a8ac853c7215b2eb8e7c2c7d0722efadaa5e19294eea41607"}
Sep 16 18:41:33.213386 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.213356    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f" event={"ID":"845d9b0a-5790-4a60-894b-f6b9adce026d","Type":"ContainerStarted","Data":"dbd6c1bf9c159cc29d1fbb275617a0d13478ba29d380c60942b36e06faf90cbd"}
Sep 16 18:41:33.254366 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.254308    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-gwwbr" podStartSLOduration=1.254289043 podStartE2EDuration="1.254289043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:33.249859443 +0000 UTC m=+121.037423616" watchObservedRunningTime="2026-09-16 18:41:33.254289043 +0000 UTC m=+121.041853199"
Sep 16 18:41:33.271936 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.271903    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 16 18:41:33.387200 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.387155    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 16 18:41:33.421123 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.421097    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 18:41:33.423209 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:33.423192    2578 projected.go:196] Error preparing data for projected volume kube-api-access-qvjtm for pod openshift-insights/insights-operator-5844b856db-vhrf8: failed to sync configmap cache: timed out waiting for the condition
Sep 16 18:41:33.423287 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:33.423274    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/306e53f3-37de-49c6-a491-f9faf67d7f2a-kube-api-access-qvjtm podName:306e53f3-37de-49c6-a491-f9faf67d7f2a nodeName:}" failed. No retries permitted until 2026-09-16 18:41:33.923255486 +0000 UTC m=+121.710819623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qvjtm" (UniqueName: "kubernetes.io/projected/306e53f3-37de-49c6-a491-f9faf67d7f2a-kube-api-access-qvjtm") pod "insights-operator-5844b856db-vhrf8" (UID: "306e53f3-37de-49c6-a491-f9faf67d7f2a") : failed to sync configmap cache: timed out waiting for the condition
Sep 16 18:41:33.573934 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.573905    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 18:41:33.662327 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.662240    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/306e53f3-37de-49c6-a491-f9faf67d7f2a-service-ca-bundle\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:33.662494 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.662369    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/306e53f3-37de-49c6-a491-f9faf67d7f2a-serving-cert\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:33.662927 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.662901    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/306e53f3-37de-49c6-a491-f9faf67d7f2a-service-ca-bundle\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:33.664978 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.664954    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/306e53f3-37de-49c6-a491-f9faf67d7f2a-serving-cert\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:33.763631 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.763587    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4vqx5\" (UID: \"54176761-3f77-4f63-ada8-c65ec07f450c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:41:33.763824 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:33.763756    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 18:41:33.763883 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:33.763838    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls podName:54176761-3f77-4f63-ada8-c65ec07f450c nodeName:}" failed. No retries permitted until 2026-09-16 18:41:35.763818474 +0000 UTC m=+123.551382628 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4vqx5" (UID: "54176761-3f77-4f63-ada8-c65ec07f450c") : secret "cluster-monitoring-operator-tls" not found
Sep 16 18:41:33.965319 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.965234    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-t6m7z\" (UID: \"19ab637e-2e0e-4e42-9a29-4da465a11b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z"
Sep 16 18:41:33.965487 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.965357    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvjtm\" (UniqueName: \"kubernetes.io/projected/306e53f3-37de-49c6-a491-f9faf67d7f2a-kube-api-access-qvjtm\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:33.965487 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:33.965407    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 18:41:33.965601 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:33.965490    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls podName:19ab637e-2e0e-4e42-9a29-4da465a11b5e nodeName:}" failed. No retries permitted until 2026-09-16 18:41:35.965467214 +0000 UTC m=+123.753031349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-t6m7z" (UID: "19ab637e-2e0e-4e42-9a29-4da465a11b5e") : secret "samples-operator-tls" not found
Sep 16 18:41:33.968386 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:33.968360    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvjtm\" (UniqueName: \"kubernetes.io/projected/306e53f3-37de-49c6-a491-f9faf67d7f2a-kube-api-access-qvjtm\") pod \"insights-operator-5844b856db-vhrf8\" (UID: \"306e53f3-37de-49c6-a491-f9faf67d7f2a\") " pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:34.065905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:34.065871    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-vr97q\" (UID: \"6d1b4b46-d618-4ceb-8e69-d034dce9b435\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"
Sep 16 18:41:34.066318 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:34.066019    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 18:41:34.066318 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:34.066086    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert podName:6d1b4b46-d618-4ceb-8e69-d034dce9b435 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:36.066071526 +0000 UTC m=+123.853635663 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-vr97q" (UID: "6d1b4b46-d618-4ceb-8e69-d034dce9b435") : secret "networking-console-plugin-cert" not found
Sep 16 18:41:34.070853 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:34.070837    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-vhrf8"
Sep 16 18:41:34.218469 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:34.218398    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9d6k" event={"ID":"6a1e4944-7d0e-4a4c-8cd0-b1a762f4798c","Type":"ContainerStarted","Data":"9af29f2d3041111e0ffe2abab088bbf403c566b37a962af4a598580102d17e6b"}
Sep 16 18:41:34.372075 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:34.372012    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-b9d6k" podStartSLOduration=2.567481628 podStartE2EDuration="3.37198855s" totalImagesPullingTime="804.506922ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:34.243680094 +0000 UTC m=+122.031244276" watchObservedRunningTime="2026-09-16 18:41:34.37198855 +0000 UTC m=+122.159552708"
Sep 16 18:41:34.372873 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:34.372847    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-vhrf8"]
Sep 16 18:41:34.730644 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:41:34.730612    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306e53f3_37de_49c6_a491_f9faf67d7f2a.slice/crio-dc61110aaad1a50bda2ffcc65f31f36ee9b2580ff01b4a3736d6298f2cd29c30 WatchSource:0}: Error finding container dc61110aaad1a50bda2ffcc65f31f36ee9b2580ff01b4a3736d6298f2cd29c30: Status 404 returned error can't find the container with id dc61110aaad1a50bda2ffcc65f31f36ee9b2580ff01b4a3736d6298f2cd29c30
Sep 16 18:41:35.224488 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:35.224460    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/0.log"
Sep 16 18:41:35.224896 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:35.224572    2578 generic.go:356] "Generic (PLEG): container finished" podID="845d9b0a-5790-4a60-894b-f6b9adce026d" containerID="aab25fa4e09fb73565b80e0fcd9108246befe87a573c4286f94ef106c9fdae1b" exitCode=255
Sep 16 18:41:35.224896 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:35.224660    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f" event={"ID":"845d9b0a-5790-4a60-894b-f6b9adce026d","Type":"ContainerDied","Data":"aab25fa4e09fb73565b80e0fcd9108246befe87a573c4286f94ef106c9fdae1b"}
Sep 16 18:41:35.224896 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:35.224883    2578 scope.go:117] "RemoveContainer" containerID="aab25fa4e09fb73565b80e0fcd9108246befe87a573c4286f94ef106c9fdae1b"
Sep 16 18:41:35.226483 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:35.226454    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-vhrf8" event={"ID":"306e53f3-37de-49c6-a491-f9faf67d7f2a","Type":"ContainerStarted","Data":"dc61110aaad1a50bda2ffcc65f31f36ee9b2580ff01b4a3736d6298f2cd29c30"}
Sep 16 18:41:35.780015 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:35.779979    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4vqx5\" (UID: \"54176761-3f77-4f63-ada8-c65ec07f450c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:41:35.780208 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:35.780138    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 18:41:35.780251 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:35.780220    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls podName:54176761-3f77-4f63-ada8-c65ec07f450c nodeName:}" failed. No retries permitted until 2026-09-16 18:41:39.780204365 +0000 UTC m=+127.567768497 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4vqx5" (UID: "54176761-3f77-4f63-ada8-c65ec07f450c") : secret "cluster-monitoring-operator-tls" not found
Sep 16 18:41:35.981004 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:35.980967    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-t6m7z\" (UID: \"19ab637e-2e0e-4e42-9a29-4da465a11b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z"
Sep 16 18:41:35.981186 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:35.981120    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 18:41:35.981249 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:35.981197    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls podName:19ab637e-2e0e-4e42-9a29-4da465a11b5e nodeName:}" failed. No retries permitted until 2026-09-16 18:41:39.981167533 +0000 UTC m=+127.768731671 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-t6m7z" (UID: "19ab637e-2e0e-4e42-9a29-4da465a11b5e") : secret "samples-operator-tls" not found
Sep 16 18:41:36.082382 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:36.082281    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-vr97q\" (UID: \"6d1b4b46-d618-4ceb-8e69-d034dce9b435\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"
Sep 16 18:41:36.082545 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:36.082445    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 18:41:36.082545 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:36.082514    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert podName:6d1b4b46-d618-4ceb-8e69-d034dce9b435 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:40.082493857 +0000 UTC m=+127.870057997 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-vr97q" (UID: "6d1b4b46-d618-4ceb-8e69-d034dce9b435") : secret "networking-console-plugin-cert" not found
Sep 16 18:41:36.231406 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:36.231375    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/1.log"
Sep 16 18:41:36.231881 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:36.231860    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/0.log"
Sep 16 18:41:36.231951 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:36.231899    2578 generic.go:356] "Generic (PLEG): container finished" podID="845d9b0a-5790-4a60-894b-f6b9adce026d" containerID="f383b2376ea7a70981181fd5e75697f237b70ecdc0683259335b1de5c608423e" exitCode=255
Sep 16 18:41:36.232005 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:36.231980    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f" event={"ID":"845d9b0a-5790-4a60-894b-f6b9adce026d","Type":"ContainerDied","Data":"f383b2376ea7a70981181fd5e75697f237b70ecdc0683259335b1de5c608423e"}
Sep 16 18:41:36.232061 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:36.232029    2578 scope.go:117] "RemoveContainer" containerID="aab25fa4e09fb73565b80e0fcd9108246befe87a573c4286f94ef106c9fdae1b"
Sep 16 18:41:36.232321 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:36.232304    2578 scope.go:117] "RemoveContainer" containerID="f383b2376ea7a70981181fd5e75697f237b70ecdc0683259335b1de5c608423e"
Sep 16 18:41:36.232514 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:36.232492    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-ttc6f_openshift-console-operator(845d9b0a-5790-4a60-894b-f6b9adce026d)\"" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f" podUID="845d9b0a-5790-4a60-894b-f6b9adce026d"
Sep 16 18:41:37.236967 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:37.236925    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-vhrf8" event={"ID":"306e53f3-37de-49c6-a491-f9faf67d7f2a","Type":"ContainerStarted","Data":"5f0a9273991b6fab4303c8cbddd40576064a779b4ecaf5ff69a90635df565de8"}
Sep 16 18:41:37.238473 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:37.238454    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/1.log"
Sep 16 18:41:37.238754 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:37.238738    2578 scope.go:117] "RemoveContainer" containerID="f383b2376ea7a70981181fd5e75697f237b70ecdc0683259335b1de5c608423e"
Sep 16 18:41:37.238886 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:37.238869    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-ttc6f_openshift-console-operator(845d9b0a-5790-4a60-894b-f6b9adce026d)\"" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f" podUID="845d9b0a-5790-4a60-894b-f6b9adce026d"
Sep 16 18:41:37.395544 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:37.395493    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-vhrf8" podStartSLOduration=4.736081889 podStartE2EDuration="6.39547469s" totalImagesPullingTime="1.659392801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:37.359162148 +0000 UTC m=+125.146726303" watchObservedRunningTime="2026-09-16 18:41:37.39547469 +0000 UTC m=+125.183038845"
Sep 16 18:41:38.941502 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:38.941467    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-brz48"]
Sep 16 18:41:38.943771 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:38.943754    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:38.949860 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:38.949560    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 18:41:38.952519 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:38.952495    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 18:41:38.952756 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:38.952741    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-blflj\""
Sep 16 18:41:39.008018 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.007977    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:39.008018 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.008023    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-crio-socket\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:39.008262 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.008040    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-data-volume\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:39.008262 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.008059    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k85fl\" (UniqueName: \"kubernetes.io/projected/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-api-access-k85fl\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:39.008262 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.008125    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:39.089787 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.089755    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-brz48"]
Sep 16 18:41:39.109489 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.109460    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:39.109666 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.109538    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:39.109666 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.109581    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-crio-socket\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:39.109666 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:39.109596    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:41:39.109666 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.109606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-data-volume\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:39.109666 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.109636    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k85fl\" (UniqueName: \"kubernetes.io/projected/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-api-access-k85fl\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:39.109666 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:39.109660    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm podName:530a3e11-7a4d-4b54-aac7-72ae0e5f3e95 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:39.609640802 +0000 UTC m=+127.397204941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm") pod "insights-runtime-extractor-brz48" (UID: "530a3e11-7a4d-4b54-aac7-72ae0e5f3e95") : configmap "kube-rbac-proxy" not found
Sep 16 18:41:39.109885 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.109710    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-crio-socket\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:39.109885 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:39.109759    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:41:39.109885 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:39.109802    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls podName:530a3e11-7a4d-4b54-aac7-72ae0e5f3e95 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:39.609790564 +0000 UTC m=+127.397354700 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls") pod "insights-runtime-extractor-brz48" (UID: "530a3e11-7a4d-4b54-aac7-72ae0e5f3e95") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:41:39.109998 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.109959    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-data-volume\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:39.130614 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.130581    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k85fl\" (UniqueName: \"kubernetes.io/projected/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-api-access-k85fl\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:39.345478 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.345400    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-xqldk"]
Sep 16 18:41:39.347556 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.347540    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-xqldk"
Sep 16 18:41:39.351525 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.351506    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 16 18:41:39.351633 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.351507    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 16 18:41:39.351633 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.351582    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-tsktv\""
Sep 16 18:41:39.352497 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.352471    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 16 18:41:39.356685 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.356664    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 16 18:41:39.389955 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.389929    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-xqldk"]
Sep 16 18:41:39.412596 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.412563    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/da4f21cf-49c2-4976-aacf-13fc7daf19ef-signing-key\") pod \"service-ca-568869fdc5-xqldk\" (UID: \"da4f21cf-49c2-4976-aacf-13fc7daf19ef\") " pod="openshift-service-ca/service-ca-568869fdc5-xqldk"
Sep 16 18:41:39.413015 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.412983    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/da4f21cf-49c2-4976-aacf-13fc7daf19ef-signing-cabundle\") pod \"service-ca-568869fdc5-xqldk\" (UID: \"da4f21cf-49c2-4976-aacf-13fc7daf19ef\") " pod="openshift-service-ca/service-ca-568869fdc5-xqldk"
Sep 16 18:41:39.413155 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.413039    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc6dl\" (UniqueName: \"kubernetes.io/projected/da4f21cf-49c2-4976-aacf-13fc7daf19ef-kube-api-access-tc6dl\") pod \"service-ca-568869fdc5-xqldk\" (UID: \"da4f21cf-49c2-4976-aacf-13fc7daf19ef\") " pod="openshift-service-ca/service-ca-568869fdc5-xqldk"
Sep 16 18:41:39.513686 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.513657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/da4f21cf-49c2-4976-aacf-13fc7daf19ef-signing-key\") pod \"service-ca-568869fdc5-xqldk\" (UID: \"da4f21cf-49c2-4976-aacf-13fc7daf19ef\") " pod="openshift-service-ca/service-ca-568869fdc5-xqldk"
Sep 16 18:41:39.513866 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.513723    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/da4f21cf-49c2-4976-aacf-13fc7daf19ef-signing-cabundle\") pod \"service-ca-568869fdc5-xqldk\" (UID: \"da4f21cf-49c2-4976-aacf-13fc7daf19ef\") " pod="openshift-service-ca/service-ca-568869fdc5-xqldk"
Sep 16 18:41:39.513866 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.513844    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc6dl\" (UniqueName: \"kubernetes.io/projected/da4f21cf-49c2-4976-aacf-13fc7daf19ef-kube-api-access-tc6dl\") pod \"service-ca-568869fdc5-xqldk\" (UID: \"da4f21cf-49c2-4976-aacf-13fc7daf19ef\") " pod="openshift-service-ca/service-ca-568869fdc5-xqldk"
Sep 16 18:41:39.514361 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.514343    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/da4f21cf-49c2-4976-aacf-13fc7daf19ef-signing-cabundle\") pod \"service-ca-568869fdc5-xqldk\" (UID: \"da4f21cf-49c2-4976-aacf-13fc7daf19ef\") " pod="openshift-service-ca/service-ca-568869fdc5-xqldk"
Sep 16 18:41:39.516295 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.516271    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/da4f21cf-49c2-4976-aacf-13fc7daf19ef-signing-key\") pod \"service-ca-568869fdc5-xqldk\" (UID: \"da4f21cf-49c2-4976-aacf-13fc7daf19ef\") " pod="openshift-service-ca/service-ca-568869fdc5-xqldk"
Sep 16 18:41:39.525487 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.525465    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc6dl\" (UniqueName: \"kubernetes.io/projected/da4f21cf-49c2-4976-aacf-13fc7daf19ef-kube-api-access-tc6dl\") pod \"service-ca-568869fdc5-xqldk\" (UID: \"da4f21cf-49c2-4976-aacf-13fc7daf19ef\") " pod="openshift-service-ca/service-ca-568869fdc5-xqldk"
Sep 16 18:41:39.614841 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.614814    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:39.615027 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.614884    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:39.615027 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:39.614957    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:41:39.615027 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:39.614963    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:41:39.615027 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:39.615011    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm podName:530a3e11-7a4d-4b54-aac7-72ae0e5f3e95 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:40.614996569 +0000 UTC m=+128.402560721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm") pod "insights-runtime-extractor-brz48" (UID: "530a3e11-7a4d-4b54-aac7-72ae0e5f3e95") : configmap "kube-rbac-proxy" not found
Sep 16 18:41:39.615027 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:39.615024    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls podName:530a3e11-7a4d-4b54-aac7-72ae0e5f3e95 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:40.615018232 +0000 UTC m=+128.402582365 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls") pod "insights-runtime-extractor-brz48" (UID: "530a3e11-7a4d-4b54-aac7-72ae0e5f3e95") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:41:39.656486 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.656448    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-xqldk"
Sep 16 18:41:39.816842 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.816812    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4vqx5\" (UID: \"54176761-3f77-4f63-ada8-c65ec07f450c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:41:39.817026 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:39.816979    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 18:41:39.817094 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:39.817059    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls podName:54176761-3f77-4f63-ada8-c65ec07f450c nodeName:}" failed. No retries permitted until 2026-09-16 18:41:47.817038497 +0000 UTC m=+135.604602636 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4vqx5" (UID: "54176761-3f77-4f63-ada8-c65ec07f450c") : secret "cluster-monitoring-operator-tls" not found
Sep 16 18:41:39.941052 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:39.940900    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-xqldk"]
Sep 16 18:41:39.944385 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:41:39.944359    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda4f21cf_49c2_4976_aacf_13fc7daf19ef.slice/crio-b0040afdd2e67b171d007a146b57186ea811049730a88b17060eefa75c6bdef2 WatchSource:0}: Error finding container b0040afdd2e67b171d007a146b57186ea811049730a88b17060eefa75c6bdef2: Status 404 returned error can't find the container with id b0040afdd2e67b171d007a146b57186ea811049730a88b17060eefa75c6bdef2
Sep 16 18:41:40.018277 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:40.018245    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-t6m7z\" (UID: \"19ab637e-2e0e-4e42-9a29-4da465a11b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z"
Sep 16 18:41:40.018467 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:40.018417    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 18:41:40.018528 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:40.018497    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls podName:19ab637e-2e0e-4e42-9a29-4da465a11b5e nodeName:}" failed. No retries permitted until 2026-09-16 18:41:48.018476453 +0000 UTC m=+135.806040587 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-t6m7z" (UID: "19ab637e-2e0e-4e42-9a29-4da465a11b5e") : secret "samples-operator-tls" not found
Sep 16 18:41:40.119337 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:40.119293    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-vr97q\" (UID: \"6d1b4b46-d618-4ceb-8e69-d034dce9b435\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"
Sep 16 18:41:40.119524 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:40.119443    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 18:41:40.119524 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:40.119518    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert podName:6d1b4b46-d618-4ceb-8e69-d034dce9b435 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:48.119499899 +0000 UTC m=+135.907064048 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-vr97q" (UID: "6d1b4b46-d618-4ceb-8e69-d034dce9b435") : secret "networking-console-plugin-cert" not found
Sep 16 18:41:40.251766 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:40.251681    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-xqldk" event={"ID":"da4f21cf-49c2-4976-aacf-13fc7daf19ef","Type":"ContainerStarted","Data":"b0040afdd2e67b171d007a146b57186ea811049730a88b17060eefa75c6bdef2"}
Sep 16 18:41:40.624521 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:40.624485    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:40.624710 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:40.624572    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:40.624710 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:40.624625    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:41:40.624710 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:40.624697    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm podName:530a3e11-7a4d-4b54-aac7-72ae0e5f3e95 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:42.624679537 +0000 UTC m=+130.412243669 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm") pod "insights-runtime-extractor-brz48" (UID: "530a3e11-7a4d-4b54-aac7-72ae0e5f3e95") : configmap "kube-rbac-proxy" not found
Sep 16 18:41:40.624710 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:40.624705    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:41:40.624878 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:40.624768    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls podName:530a3e11-7a4d-4b54-aac7-72ae0e5f3e95 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:42.624752374 +0000 UTC m=+130.412316513 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls") pod "insights-runtime-extractor-brz48" (UID: "530a3e11-7a4d-4b54-aac7-72ae0e5f3e95") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:41:41.188411 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:41.188382    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-57l9p_d96fa3e0-be40-4f37-99be-57261dce04d1/dns-node-resolver/0.log"
Sep 16 18:41:41.431301 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:41.431256    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs\") pod \"network-metrics-daemon-282m9\" (UID: \"87684818-77c6-45bc-bee8-61a206348549\") " pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:41:41.431491 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:41.431423    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 18:41:41.431554 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:41.431494    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs podName:87684818-77c6-45bc-bee8-61a206348549 nodeName:}" failed. No retries permitted until 2026-09-16 18:43:43.431477465 +0000 UTC m=+251.219041611 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs") pod "network-metrics-daemon-282m9" (UID: "87684818-77c6-45bc-bee8-61a206348549") : secret "metrics-daemon-secret" not found
Sep 16 18:41:42.190887 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:42.190858    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-478xv_fae250ed-5b59-4388-af13-fcc8d0446e5d/node-ca/0.log"
Sep 16 18:41:42.260526 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:42.260488    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-xqldk" event={"ID":"da4f21cf-49c2-4976-aacf-13fc7daf19ef","Type":"ContainerStarted","Data":"881b6256d3bf8335e971bca6b61941efb468962e9e327bf8946ab86050bbbe15"}
Sep 16 18:41:42.299834 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:42.299783    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-xqldk" podStartSLOduration=1.764989014 podStartE2EDuration="3.299763144s" totalImagesPullingTime="1.53477413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:42.298810114 +0000 UTC m=+130.086374271" watchObservedRunningTime="2026-09-16 18:41:42.299763144 +0000 UTC m=+130.087327299"
Sep 16 18:41:42.574417 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:42.574330    2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:41:42.574417 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:42.574398    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:41:42.574832 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:42.574818    2578 scope.go:117] "RemoveContainer" containerID="f383b2376ea7a70981181fd5e75697f237b70ecdc0683259335b1de5c608423e"
Sep 16 18:41:42.575018 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:42.574997    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-ttc6f_openshift-console-operator(845d9b0a-5790-4a60-894b-f6b9adce026d)\"" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f" podUID="845d9b0a-5790-4a60-894b-f6b9adce026d"
Sep 16 18:41:42.640830 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:42.640779    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:42.641028 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:42.640908    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:42.641028 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:42.640923    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:41:42.641028 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:42.640980    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:41:42.641028 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:42.641013    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls podName:530a3e11-7a4d-4b54-aac7-72ae0e5f3e95 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:46.640992875 +0000 UTC m=+134.428557028 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls") pod "insights-runtime-extractor-brz48" (UID: "530a3e11-7a4d-4b54-aac7-72ae0e5f3e95") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:41:42.641028 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:42.641035    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm podName:530a3e11-7a4d-4b54-aac7-72ae0e5f3e95 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:46.641025178 +0000 UTC m=+134.428589318 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm") pod "insights-runtime-extractor-brz48" (UID: "530a3e11-7a4d-4b54-aac7-72ae0e5f3e95") : configmap "kube-rbac-proxy" not found
Sep 16 18:41:42.996777 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:42.996736    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-h6s5l_f21dc980-1bae-414a-be6b-7145bcc7c7f5/serve-healthcheck-canary/0.log"
Sep 16 18:41:46.674923 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:46.674867    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:46.675383 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:46.674966    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:46.675383 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:46.675018    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:41:46.675383 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:46.675036    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:41:46.675383 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:46.675081    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm podName:530a3e11-7a4d-4b54-aac7-72ae0e5f3e95 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:54.675067967 +0000 UTC m=+142.462632114 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm") pod "insights-runtime-extractor-brz48" (UID: "530a3e11-7a4d-4b54-aac7-72ae0e5f3e95") : configmap "kube-rbac-proxy" not found
Sep 16 18:41:46.675383 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:46.675092    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls podName:530a3e11-7a4d-4b54-aac7-72ae0e5f3e95 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:54.675087021 +0000 UTC m=+142.462651154 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls") pod "insights-runtime-extractor-brz48" (UID: "530a3e11-7a4d-4b54-aac7-72ae0e5f3e95") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:41:47.884646 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:47.884606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4vqx5\" (UID: \"54176761-3f77-4f63-ada8-c65ec07f450c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:41:47.885028 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:47.884765    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 18:41:47.885028 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:47.884842    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls podName:54176761-3f77-4f63-ada8-c65ec07f450c nodeName:}" failed. No retries permitted until 2026-09-16 18:42:03.884822643 +0000 UTC m=+151.672386778 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4vqx5" (UID: "54176761-3f77-4f63-ada8-c65ec07f450c") : secret "cluster-monitoring-operator-tls" not found
Sep 16 18:41:48.086745 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:48.086709    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-t6m7z\" (UID: \"19ab637e-2e0e-4e42-9a29-4da465a11b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z"
Sep 16 18:41:48.089093 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:48.089072    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/19ab637e-2e0e-4e42-9a29-4da465a11b5e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-t6m7z\" (UID: \"19ab637e-2e0e-4e42-9a29-4da465a11b5e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z"
Sep 16 18:41:48.171836 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:48.171753    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pgdkz\""
Sep 16 18:41:48.179890 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:48.179866    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z"
Sep 16 18:41:48.188046 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:48.188021    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-vr97q\" (UID: \"6d1b4b46-d618-4ceb-8e69-d034dce9b435\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"
Sep 16 18:41:48.188266 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:48.188180    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 16 18:41:48.188266 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:48.188243    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert podName:6d1b4b46-d618-4ceb-8e69-d034dce9b435 nodeName:}" failed. No retries permitted until 2026-09-16 18:42:04.188226452 +0000 UTC m=+151.975790585 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-vr97q" (UID: "6d1b4b46-d618-4ceb-8e69-d034dce9b435") : secret "networking-console-plugin-cert" not found
Sep 16 18:41:48.458240 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:48.458153    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z"]
Sep 16 18:41:49.293892 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:49.293836    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z" event={"ID":"19ab637e-2e0e-4e42-9a29-4da465a11b5e","Type":"ContainerStarted","Data":"71584e92afec26ec0036fe64b72e8a8cbcf654a42aecea45f9a3281dd8971f2a"}
Sep 16 18:41:50.301828 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:50.301787    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z" event={"ID":"19ab637e-2e0e-4e42-9a29-4da465a11b5e","Type":"ContainerStarted","Data":"7501c15bdfa7a187c782723b680df67ed1e9bf6e8287395fdfa03db609e74ed3"}
Sep 16 18:41:50.302224 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:50.301873    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z" event={"ID":"19ab637e-2e0e-4e42-9a29-4da465a11b5e","Type":"ContainerStarted","Data":"9ecfa5536074de6b9562518ed1c71940331541b51dd835b74152dbdc36ead911"}
Sep 16 18:41:50.338326 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:50.338264    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-t6m7z" podStartSLOduration=16.760389633 podStartE2EDuration="18.338243254s" totalImagesPullingTime="1.577853621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:50.33819159 +0000 UTC m=+138.125755792" watchObservedRunningTime="2026-09-16 18:41:50.338243254 +0000 UTC m=+138.125807413"
Sep 16 18:41:54.742806 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:54.742762    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:54.743221 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:54.742862    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:54.743431 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:54.743410    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:54.745122 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:54.745106    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/530a3e11-7a4d-4b54-aac7-72ae0e5f3e95-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-brz48\" (UID: \"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95\") " pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:54.851782 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:54.851748    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-brz48"
Sep 16 18:41:55.126538 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:55.126516    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-brz48"]
Sep 16 18:41:55.128748 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:41:55.128718    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod530a3e11_7a4d_4b54_aac7_72ae0e5f3e95.slice/crio-1b31bc5bbddbc97963bd0ee8452aa83ac4d2e112d05be897994fd6e5e2691936 WatchSource:0}: Error finding container 1b31bc5bbddbc97963bd0ee8452aa83ac4d2e112d05be897994fd6e5e2691936: Status 404 returned error can't find the container with id 1b31bc5bbddbc97963bd0ee8452aa83ac4d2e112d05be897994fd6e5e2691936
Sep 16 18:41:55.319403 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:55.319367    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-brz48" event={"ID":"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95","Type":"ContainerStarted","Data":"99780c947af27ce9ccd5e9158f70025e6412b434ff77cbccf8fb7ec6bd4df2fc"}
Sep 16 18:41:55.319560 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:55.319427    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-brz48" event={"ID":"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95","Type":"ContainerStarted","Data":"1b31bc5bbddbc97963bd0ee8452aa83ac4d2e112d05be897994fd6e5e2691936"}
Sep 16 18:41:55.703420 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:55.703384    2578 scope.go:117] "RemoveContainer" containerID="f383b2376ea7a70981181fd5e75697f237b70ecdc0683259335b1de5c608423e"
Sep 16 18:41:56.324765 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:56.324733    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 18:41:56.325289 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:56.325267    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/1.log"
Sep 16 18:41:56.325354 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:56.325314    2578 generic.go:356] "Generic (PLEG): container finished" podID="845d9b0a-5790-4a60-894b-f6b9adce026d" containerID="7d9930c95a2d36cfe3ee8a39a7e546c5d23417e64fea20d63288f8bc4a62acaf" exitCode=255
Sep 16 18:41:56.325431 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:56.325402    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f" event={"ID":"845d9b0a-5790-4a60-894b-f6b9adce026d","Type":"ContainerDied","Data":"7d9930c95a2d36cfe3ee8a39a7e546c5d23417e64fea20d63288f8bc4a62acaf"}
Sep 16 18:41:56.325529 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:56.325513    2578 scope.go:117] "RemoveContainer" containerID="f383b2376ea7a70981181fd5e75697f237b70ecdc0683259335b1de5c608423e"
Sep 16 18:41:56.325904 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:56.325816    2578 scope.go:117] "RemoveContainer" containerID="7d9930c95a2d36cfe3ee8a39a7e546c5d23417e64fea20d63288f8bc4a62acaf"
Sep 16 18:41:56.326035 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:41:56.326007    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-ttc6f_openshift-console-operator(845d9b0a-5790-4a60-894b-f6b9adce026d)\"" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f" podUID="845d9b0a-5790-4a60-894b-f6b9adce026d"
Sep 16 18:41:56.327584 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:56.327560    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-brz48" event={"ID":"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95","Type":"ContainerStarted","Data":"06660a7083a82dc588f383ebc655451d76a734a324a4a6efbcd458fbbe348865"}
Sep 16 18:41:57.332937 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:57.332848    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-brz48" event={"ID":"530a3e11-7a4d-4b54-aac7-72ae0e5f3e95","Type":"ContainerStarted","Data":"a32ced5a74d45f7a3f57e7cd88d5ae6f1825f0620dac9467c36991397f309053"}
Sep 16 18:41:57.334357 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:57.334336    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 18:41:57.366683 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:41:57.366633    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-brz48" podStartSLOduration=17.658147553 podStartE2EDuration="19.36661762s" totalImagesPullingTime="1.708470067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:41:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:57.3658611 +0000 UTC m=+145.153425275" watchObservedRunningTime="2026-09-16 18:41:57.36661762 +0000 UTC m=+145.154181772"
Sep 16 18:42:02.574281 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:02.574245    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:42:02.574660 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:02.574297    2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:42:02.574660 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:02.574576    2578 scope.go:117] "RemoveContainer" containerID="7d9930c95a2d36cfe3ee8a39a7e546c5d23417e64fea20d63288f8bc4a62acaf"
Sep 16 18:42:02.574745 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:42:02.574725    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-ttc6f_openshift-console-operator(845d9b0a-5790-4a60-894b-f6b9adce026d)\"" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f" podUID="845d9b0a-5790-4a60-894b-f6b9adce026d"
Sep 16 18:42:03.913232 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:03.913197    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4vqx5\" (UID: \"54176761-3f77-4f63-ada8-c65ec07f450c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:42:03.915707 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:03.915678    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54176761-3f77-4f63-ada8-c65ec07f450c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4vqx5\" (UID: \"54176761-3f77-4f63-ada8-c65ec07f450c\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:42:04.215671 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.215575    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-vr97q\" (UID: \"6d1b4b46-d618-4ceb-8e69-d034dce9b435\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"
Sep 16 18:42:04.217980 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.217961    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6d1b4b46-d618-4ceb-8e69-d034dce9b435-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-vr97q\" (UID: \"6d1b4b46-d618-4ceb-8e69-d034dce9b435\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"
Sep 16 18:42:04.222387 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.222362    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mz8qf\""
Sep 16 18:42:04.227358 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.227344    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"
Sep 16 18:42:04.505341 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.505257    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gdrdg\""
Sep 16 18:42:04.511933 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.511913    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"
Sep 16 18:42:04.568911 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:42:04.568882    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54176761_3f77_4f63_ada8_c65ec07f450c.slice/crio-31f53e961b273ed55a90afc541381f19bff12304ab42c80b888ad91710e844a9 WatchSource:0}: Error finding container 31f53e961b273ed55a90afc541381f19bff12304ab42c80b888ad91710e844a9: Status 404 returned error can't find the container with id 31f53e961b273ed55a90afc541381f19bff12304ab42c80b888ad91710e844a9
Sep 16 18:42:04.570820 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.570705    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5"]
Sep 16 18:42:04.605026 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.604996    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-68974dbd59-lgnkg"]
Sep 16 18:42:04.607560 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.607544    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.618305 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.618284    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 16 18:42:04.618533 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.618520    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tm7mf\""
Sep 16 18:42:04.618600 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.618584    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 16 18:42:04.625510 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.625493    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 16 18:42:04.663744 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.663715    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68974dbd59-lgnkg"]
Sep 16 18:42:04.692237 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.692204    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 16 18:42:04.718324 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.718292    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9cebc0d4-b215-494b-b236-32640184278b-installation-pull-secrets\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.718324 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.718324    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrzlr\" (UniqueName: \"kubernetes.io/projected/9cebc0d4-b215-494b-b236-32640184278b-kube-api-access-lrzlr\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.718505 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.718360    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cebc0d4-b215-494b-b236-32640184278b-registry-tls\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.718505 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.718385    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9cebc0d4-b215-494b-b236-32640184278b-image-registry-private-configuration\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.718505 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.718417    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cebc0d4-b215-494b-b236-32640184278b-bound-sa-token\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.718505 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.718474    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cebc0d4-b215-494b-b236-32640184278b-trusted-ca\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.718505 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.718492    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9cebc0d4-b215-494b-b236-32640184278b-registry-certificates\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.718671 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.718520    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9cebc0d4-b215-494b-b236-32640184278b-ca-trust-extracted\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.795629 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.795557    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q"]
Sep 16 18:42:04.798154 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:42:04.798119    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d1b4b46_d618_4ceb_8e69_d034dce9b435.slice/crio-e3cee1192f8f02d9cd22118a9c2905b86a94853856b983194de9d1df4d3ed449 WatchSource:0}: Error finding container e3cee1192f8f02d9cd22118a9c2905b86a94853856b983194de9d1df4d3ed449: Status 404 returned error can't find the container with id e3cee1192f8f02d9cd22118a9c2905b86a94853856b983194de9d1df4d3ed449
Sep 16 18:42:04.819850 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.819828    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9cebc0d4-b215-494b-b236-32640184278b-installation-pull-secrets\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.819958 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.819859    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrzlr\" (UniqueName: \"kubernetes.io/projected/9cebc0d4-b215-494b-b236-32640184278b-kube-api-access-lrzlr\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.819958 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.819891    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cebc0d4-b215-494b-b236-32640184278b-registry-tls\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.819958 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.819917    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9cebc0d4-b215-494b-b236-32640184278b-image-registry-private-configuration\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.819958 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.819943    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cebc0d4-b215-494b-b236-32640184278b-bound-sa-token\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.820191 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.819986    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cebc0d4-b215-494b-b236-32640184278b-trusted-ca\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.820191 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.820004    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9cebc0d4-b215-494b-b236-32640184278b-registry-certificates\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.820191 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.820038    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9cebc0d4-b215-494b-b236-32640184278b-ca-trust-extracted\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.820409 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.820381    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9cebc0d4-b215-494b-b236-32640184278b-ca-trust-extracted\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.820823 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.820798    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9cebc0d4-b215-494b-b236-32640184278b-trusted-ca\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.820924 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.820886    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9cebc0d4-b215-494b-b236-32640184278b-registry-certificates\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.822458 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.822438    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9cebc0d4-b215-494b-b236-32640184278b-installation-pull-secrets\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.822539 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.822453    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9cebc0d4-b215-494b-b236-32640184278b-image-registry-private-configuration\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.822539 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.822493    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9cebc0d4-b215-494b-b236-32640184278b-registry-tls\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.854438 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.854410    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cebc0d4-b215-494b-b236-32640184278b-bound-sa-token\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.858503 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.858482    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrzlr\" (UniqueName: \"kubernetes.io/projected/9cebc0d4-b215-494b-b236-32640184278b-kube-api-access-lrzlr\") pod \"image-registry-68974dbd59-lgnkg\" (UID: \"9cebc0d4-b215-494b-b236-32640184278b\") " pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:04.916585 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:04.916558    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:05.235973 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:05.235939    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-68974dbd59-lgnkg"]
Sep 16 18:42:05.239069 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:42:05.239034    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cebc0d4_b215_494b_b236_32640184278b.slice/crio-e7e63aa49f0256ae222a5a9b0a407f66bf0ad95daf8cea2ba04b0523fb96c522 WatchSource:0}: Error finding container e7e63aa49f0256ae222a5a9b0a407f66bf0ad95daf8cea2ba04b0523fb96c522: Status 404 returned error can't find the container with id e7e63aa49f0256ae222a5a9b0a407f66bf0ad95daf8cea2ba04b0523fb96c522
Sep 16 18:42:05.362970 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:05.362898    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q" event={"ID":"6d1b4b46-d618-4ceb-8e69-d034dce9b435","Type":"ContainerStarted","Data":"e3cee1192f8f02d9cd22118a9c2905b86a94853856b983194de9d1df4d3ed449"}
Sep 16 18:42:05.364544 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:05.364514    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5" event={"ID":"54176761-3f77-4f63-ada8-c65ec07f450c","Type":"ContainerStarted","Data":"31f53e961b273ed55a90afc541381f19bff12304ab42c80b888ad91710e844a9"}
Sep 16 18:42:05.366875 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:05.366765    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68974dbd59-lgnkg" event={"ID":"9cebc0d4-b215-494b-b236-32640184278b","Type":"ContainerStarted","Data":"b87ed77fa9248bfa7f3a201475744018d611dce099094ccd40d87fed6f1359db"}
Sep 16 18:42:05.366875 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:05.366809    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-68974dbd59-lgnkg" event={"ID":"9cebc0d4-b215-494b-b236-32640184278b","Type":"ContainerStarted","Data":"e7e63aa49f0256ae222a5a9b0a407f66bf0ad95daf8cea2ba04b0523fb96c522"}
Sep 16 18:42:05.366875 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:05.366848    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:07.373787 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:07.373747    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q" event={"ID":"6d1b4b46-d618-4ceb-8e69-d034dce9b435","Type":"ContainerStarted","Data":"cd916ad088621df1ca3852178fff024c20fa8e9ec2fa523eba2edb70c0820434"}
Sep 16 18:42:07.375341 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:07.375318    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5" event={"ID":"54176761-3f77-4f63-ada8-c65ec07f450c","Type":"ContainerStarted","Data":"97af1b4d018c1eb3f24360762c3500ba05a5e9495a1497d5da7da2bd534d4361"}
Sep 16 18:42:07.401841 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:07.401792    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-vr97q" podStartSLOduration=33.824325387 podStartE2EDuration="35.401778175s" totalImagesPullingTime="1.577452788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:07.400663393 +0000 UTC m=+155.188227548" watchObservedRunningTime="2026-09-16 18:42:07.401778175 +0000 UTC m=+155.189342329"
Sep 16 18:42:07.402368 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:07.402329    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-68974dbd59-lgnkg" podStartSLOduration=3.402317352 podStartE2EDuration="3.402317352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:05.446079784 +0000 UTC m=+153.233643940" watchObservedRunningTime="2026-09-16 18:42:07.402317352 +0000 UTC m=+155.189881508"
Sep 16 18:42:09.171326 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:42:09.171280    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-crt2q" podUID="6fbdb2ba-c71f-47e6-bb39-1f2aa806df61"
Sep 16 18:42:09.383965 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:09.383925    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-crt2q"
Sep 16 18:42:10.724609 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:42:10.724568    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-282m9" podUID="87684818-77c6-45bc-bee8-61a206348549"
Sep 16 18:42:12.377267 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.377216    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4vqx5" podStartSLOduration=38.566691389 podStartE2EDuration="40.377164944s" totalImagesPullingTime="1.810473555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:07.437355914 +0000 UTC m=+155.224920087" watchObservedRunningTime="2026-09-16 18:42:12.377164944 +0000 UTC m=+160.164729102"
Sep 16 18:42:12.377859 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.377840    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-l874g"]
Sep 16 18:42:12.381219 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.381198    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g"
Sep 16 18:42:12.385129 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.385109    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 16 18:42:12.385827 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.385811    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 16 18:42:12.385893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.385866    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-p4cjv\""
Sep 16 18:42:12.386162 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.386147    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 18:42:12.393198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.393142    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-l874g"]
Sep 16 18:42:12.481687 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.481650    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25bcf7ce-8826-435a-8c06-7a229e704bf3-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-l874g\" (UID: \"25bcf7ce-8826-435a-8c06-7a229e704bf3\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g"
Sep 16 18:42:12.481870 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.481695    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25bcf7ce-8826-435a-8c06-7a229e704bf3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-l874g\" (UID: \"25bcf7ce-8826-435a-8c06-7a229e704bf3\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g"
Sep 16 18:42:12.481870 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.481728    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72fl5\" (UniqueName: \"kubernetes.io/projected/25bcf7ce-8826-435a-8c06-7a229e704bf3-kube-api-access-72fl5\") pod \"prometheus-operator-68bcd7f987-l874g\" (UID: \"25bcf7ce-8826-435a-8c06-7a229e704bf3\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g"
Sep 16 18:42:12.481870 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.481814    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/25bcf7ce-8826-435a-8c06-7a229e704bf3-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-l874g\" (UID: \"25bcf7ce-8826-435a-8c06-7a229e704bf3\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g"
Sep 16 18:42:12.582608 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.582579    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72fl5\" (UniqueName: \"kubernetes.io/projected/25bcf7ce-8826-435a-8c06-7a229e704bf3-kube-api-access-72fl5\") pod \"prometheus-operator-68bcd7f987-l874g\" (UID: \"25bcf7ce-8826-435a-8c06-7a229e704bf3\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g"
Sep 16 18:42:12.582730 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.582648    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/25bcf7ce-8826-435a-8c06-7a229e704bf3-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-l874g\" (UID: \"25bcf7ce-8826-435a-8c06-7a229e704bf3\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g"
Sep 16 18:42:12.582843 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.582815    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25bcf7ce-8826-435a-8c06-7a229e704bf3-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-l874g\" (UID: \"25bcf7ce-8826-435a-8c06-7a229e704bf3\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g"
Sep 16 18:42:12.582907 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.582882    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25bcf7ce-8826-435a-8c06-7a229e704bf3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-l874g\" (UID: \"25bcf7ce-8826-435a-8c06-7a229e704bf3\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g"
Sep 16 18:42:12.583475 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.583452    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25bcf7ce-8826-435a-8c06-7a229e704bf3-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-l874g\" (UID: \"25bcf7ce-8826-435a-8c06-7a229e704bf3\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g"
Sep 16 18:42:12.585024 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.584998    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/25bcf7ce-8826-435a-8c06-7a229e704bf3-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-l874g\" (UID: \"25bcf7ce-8826-435a-8c06-7a229e704bf3\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g"
Sep 16 18:42:12.585142 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.585120    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25bcf7ce-8826-435a-8c06-7a229e704bf3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-l874g\" (UID: \"25bcf7ce-8826-435a-8c06-7a229e704bf3\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g"
Sep 16 18:42:12.597009 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.596989    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72fl5\" (UniqueName: \"kubernetes.io/projected/25bcf7ce-8826-435a-8c06-7a229e704bf3-kube-api-access-72fl5\") pod \"prometheus-operator-68bcd7f987-l874g\" (UID: \"25bcf7ce-8826-435a-8c06-7a229e704bf3\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g"
Sep 16 18:42:12.690138 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.690056    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g"
Sep 16 18:42:12.962942 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:12.962916    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-l874g"]
Sep 16 18:42:12.964799 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:42:12.964774    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25bcf7ce_8826_435a_8c06_7a229e704bf3.slice/crio-6e59aaa7ff7d15f0bfbd92107977ecc8abc9dc7b7345550455cb4f91df94c585 WatchSource:0}: Error finding container 6e59aaa7ff7d15f0bfbd92107977ecc8abc9dc7b7345550455cb4f91df94c585: Status 404 returned error can't find the container with id 6e59aaa7ff7d15f0bfbd92107977ecc8abc9dc7b7345550455cb4f91df94c585
Sep 16 18:42:13.400422 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:13.400383    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g" event={"ID":"25bcf7ce-8826-435a-8c06-7a229e704bf3","Type":"ContainerStarted","Data":"6e59aaa7ff7d15f0bfbd92107977ecc8abc9dc7b7345550455cb4f91df94c585"}
Sep 16 18:42:14.095474 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:14.095435    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:42:14.097781 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:14.097752    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6fbdb2ba-c71f-47e6-bb39-1f2aa806df61-metrics-tls\") pod \"dns-default-crt2q\" (UID: \"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61\") " pod="openshift-dns/dns-default-crt2q"
Sep 16 18:42:14.188035 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:14.188008    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-24cbm\""
Sep 16 18:42:14.195836 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:14.195808    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-crt2q"
Sep 16 18:42:14.406436 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:14.406385    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g" event={"ID":"25bcf7ce-8826-435a-8c06-7a229e704bf3","Type":"ContainerStarted","Data":"043e940ad64157ec751814f143fd6f74ddeeed0411d9aa747a138753280bc7ad"}
Sep 16 18:42:14.406865 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:14.406486    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g" event={"ID":"25bcf7ce-8826-435a-8c06-7a229e704bf3","Type":"ContainerStarted","Data":"dd3da4039cb3a8d35002abab4009d7aa646711007b8df5ecfbacf22e9b20a4f6"}
Sep 16 18:42:14.490777 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:14.490697    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l874g" podStartSLOduration=1.3355524540000001 podStartE2EDuration="2.490670263s" totalImagesPullingTime="1.155117809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:14.429848155 +0000 UTC m=+162.217412311" watchObservedRunningTime="2026-09-16 18:42:14.490670263 +0000 UTC m=+162.278234419"
Sep 16 18:42:14.491452 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:14.491431    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-crt2q"]
Sep 16 18:42:14.495038 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:42:14.495009    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fbdb2ba_c71f_47e6_bb39_1f2aa806df61.slice/crio-41c7c7d3e32d32cd7a5c9eddb7ccc8cacb8022cfb7915ad8d7d4bd3fe044f4d5 WatchSource:0}: Error finding container 41c7c7d3e32d32cd7a5c9eddb7ccc8cacb8022cfb7915ad8d7d4bd3fe044f4d5: Status 404 returned error can't find the container with id 41c7c7d3e32d32cd7a5c9eddb7ccc8cacb8022cfb7915ad8d7d4bd3fe044f4d5
Sep 16 18:42:14.703309 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:14.703232    2578 scope.go:117] "RemoveContainer" containerID="7d9930c95a2d36cfe3ee8a39a7e546c5d23417e64fea20d63288f8bc4a62acaf"
Sep 16 18:42:14.703444 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:42:14.703418    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-ttc6f_openshift-console-operator(845d9b0a-5790-4a60-894b-f6b9adce026d)\"" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f" podUID="845d9b0a-5790-4a60-894b-f6b9adce026d"
Sep 16 18:42:15.411672 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:15.411627    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-crt2q" event={"ID":"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61","Type":"ContainerStarted","Data":"41c7c7d3e32d32cd7a5c9eddb7ccc8cacb8022cfb7915ad8d7d4bd3fe044f4d5"}
Sep 16 18:42:16.417922 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:16.417887    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-crt2q" event={"ID":"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61","Type":"ContainerStarted","Data":"391b4a9a11cb484e562cece7b0d961893002900ff5651565d17d58763cc793a7"}
Sep 16 18:42:16.418324 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:16.417939    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-crt2q" event={"ID":"6fbdb2ba-c71f-47e6-bb39-1f2aa806df61","Type":"ContainerStarted","Data":"ddee930f4aa364be80fddbb8ff18c9af0f02b4799e2f8f0bc24805c5ce7db1b7"}
Sep 16 18:42:16.418324 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:16.418015    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-crt2q"
Sep 16 18:42:17.450693 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.450631    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-crt2q" podStartSLOduration=130.0242562 podStartE2EDuration="2m11.450605659s" totalImagesPullingTime="1.426349459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:16.461708596 +0000 UTC m=+164.249272750" watchObservedRunningTime="2026-09-16 18:42:17.450605659 +0000 UTC m=+165.238169815"
Sep 16 18:42:17.451408 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.451388    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-clkzl"]
Sep 16 18:42:17.457320 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.457292    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.466336 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.466315    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 18:42:17.466435 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.466315    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 18:42:17.466652 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.466638    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 18:42:17.477307 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.477283    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"]
Sep 16 18:42:17.478570 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.478554    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xrql4\""
Sep 16 18:42:17.479366 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.479351    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"
Sep 16 18:42:17.482769 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.482750    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 16 18:42:17.483138 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.483116    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 16 18:42:17.483248 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.483125    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-88dnn\""
Sep 16 18:42:17.500418 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.500391    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"]
Sep 16 18:42:17.624089 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.624054    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/df0ff542-aa25-4096-b1d5-3064c5b308b3-root\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.624291 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.624097    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df0ff542-aa25-4096-b1d5-3064c5b308b3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.624291 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.624150    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/df0ff542-aa25-4096-b1d5-3064c5b308b3-node-exporter-textfile\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.624291 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.624205    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/df0ff542-aa25-4096-b1d5-3064c5b308b3-node-exporter-accelerators-collector-config\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.624291 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.624238    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b8172967-90c2-4159-9b6d-b0838c6703af-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-7jfq8\" (UID: \"b8172967-90c2-4159-9b6d-b0838c6703af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"
Sep 16 18:42:17.624291 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.624254    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cjz4\" (UniqueName: \"kubernetes.io/projected/b8172967-90c2-4159-9b6d-b0838c6703af-kube-api-access-6cjz4\") pod \"openshift-state-metrics-67dcf8c979-7jfq8\" (UID: \"b8172967-90c2-4159-9b6d-b0838c6703af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"
Sep 16 18:42:17.624291 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.624280    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/df0ff542-aa25-4096-b1d5-3064c5b308b3-node-exporter-wtmp\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.624577 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.624363    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8172967-90c2-4159-9b6d-b0838c6703af-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-7jfq8\" (UID: \"b8172967-90c2-4159-9b6d-b0838c6703af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"
Sep 16 18:42:17.624577 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.624393    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/df0ff542-aa25-4096-b1d5-3064c5b308b3-node-exporter-tls\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.624577 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.624412    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdd6v\" (UniqueName: \"kubernetes.io/projected/df0ff542-aa25-4096-b1d5-3064c5b308b3-kube-api-access-xdd6v\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.624577 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.624446    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8172967-90c2-4159-9b6d-b0838c6703af-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-7jfq8\" (UID: \"b8172967-90c2-4159-9b6d-b0838c6703af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"
Sep 16 18:42:17.624577 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.624509    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df0ff542-aa25-4096-b1d5-3064c5b308b3-metrics-client-ca\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.624577 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.624546    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df0ff542-aa25-4096-b1d5-3064c5b308b3-sys\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.725550 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.725454    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8172967-90c2-4159-9b6d-b0838c6703af-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-7jfq8\" (UID: \"b8172967-90c2-4159-9b6d-b0838c6703af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"
Sep 16 18:42:17.725550 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.725505    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df0ff542-aa25-4096-b1d5-3064c5b308b3-metrics-client-ca\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.725784 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.725552    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df0ff542-aa25-4096-b1d5-3064c5b308b3-sys\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.725784 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.725576    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/df0ff542-aa25-4096-b1d5-3064c5b308b3-root\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.725784 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.725599    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df0ff542-aa25-4096-b1d5-3064c5b308b3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.725784 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.725641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/df0ff542-aa25-4096-b1d5-3064c5b308b3-node-exporter-textfile\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.725784 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.725669    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/df0ff542-aa25-4096-b1d5-3064c5b308b3-sys\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.725784 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.725671    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/df0ff542-aa25-4096-b1d5-3064c5b308b3-root\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.725784 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.725745    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/df0ff542-aa25-4096-b1d5-3064c5b308b3-node-exporter-accelerators-collector-config\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.726128 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.725787    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b8172967-90c2-4159-9b6d-b0838c6703af-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-7jfq8\" (UID: \"b8172967-90c2-4159-9b6d-b0838c6703af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"
Sep 16 18:42:17.726128 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.725818    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cjz4\" (UniqueName: \"kubernetes.io/projected/b8172967-90c2-4159-9b6d-b0838c6703af-kube-api-access-6cjz4\") pod \"openshift-state-metrics-67dcf8c979-7jfq8\" (UID: \"b8172967-90c2-4159-9b6d-b0838c6703af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"
Sep 16 18:42:17.726128 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.725859    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/df0ff542-aa25-4096-b1d5-3064c5b308b3-node-exporter-wtmp\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.726128 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.725964    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/df0ff542-aa25-4096-b1d5-3064c5b308b3-node-exporter-textfile\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.726128 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.725989    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/df0ff542-aa25-4096-b1d5-3064c5b308b3-node-exporter-wtmp\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.726128 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.726021    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8172967-90c2-4159-9b6d-b0838c6703af-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-7jfq8\" (UID: \"b8172967-90c2-4159-9b6d-b0838c6703af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"
Sep 16 18:42:17.726128 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.726048    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/df0ff542-aa25-4096-b1d5-3064c5b308b3-node-exporter-tls\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.726128 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.726073    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdd6v\" (UniqueName: \"kubernetes.io/projected/df0ff542-aa25-4096-b1d5-3064c5b308b3-kube-api-access-xdd6v\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.726846 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.726822    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b8172967-90c2-4159-9b6d-b0838c6703af-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-7jfq8\" (UID: \"b8172967-90c2-4159-9b6d-b0838c6703af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"
Sep 16 18:42:17.726950 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.726927    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/df0ff542-aa25-4096-b1d5-3064c5b308b3-node-exporter-accelerators-collector-config\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.727002 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.726951    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df0ff542-aa25-4096-b1d5-3064c5b308b3-metrics-client-ca\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.728270 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.728246    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/df0ff542-aa25-4096-b1d5-3064c5b308b3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.728356 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.728273    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b8172967-90c2-4159-9b6d-b0838c6703af-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-7jfq8\" (UID: \"b8172967-90c2-4159-9b6d-b0838c6703af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"
Sep 16 18:42:17.728632 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.728605    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b8172967-90c2-4159-9b6d-b0838c6703af-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-7jfq8\" (UID: \"b8172967-90c2-4159-9b6d-b0838c6703af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"
Sep 16 18:42:17.728912 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.728890    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/df0ff542-aa25-4096-b1d5-3064c5b308b3-node-exporter-tls\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.748899 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.748862    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdd6v\" (UniqueName: \"kubernetes.io/projected/df0ff542-aa25-4096-b1d5-3064c5b308b3-kube-api-access-xdd6v\") pod \"node-exporter-clkzl\" (UID: \"df0ff542-aa25-4096-b1d5-3064c5b308b3\") " pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.760598 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.760560    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cjz4\" (UniqueName: \"kubernetes.io/projected/b8172967-90c2-4159-9b6d-b0838c6703af-kube-api-access-6cjz4\") pod \"openshift-state-metrics-67dcf8c979-7jfq8\" (UID: \"b8172967-90c2-4159-9b6d-b0838c6703af\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"
Sep 16 18:42:17.768351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.768319    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-clkzl"
Sep 16 18:42:17.779076 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:42:17.779046    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf0ff542_aa25_4096_b1d5_3064c5b308b3.slice/crio-c94e95e3dde87f2af5eea809495064bd4539326327e9e206d53082e0bb7eccec WatchSource:0}: Error finding container c94e95e3dde87f2af5eea809495064bd4539326327e9e206d53082e0bb7eccec: Status 404 returned error can't find the container with id c94e95e3dde87f2af5eea809495064bd4539326327e9e206d53082e0bb7eccec
Sep 16 18:42:17.789463 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:17.789435    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"
Sep 16 18:42:18.090252 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:18.090216    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8"]
Sep 16 18:42:18.092978 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:42:18.092947    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8172967_90c2_4159_9b6d_b0838c6703af.slice/crio-0bd70847d3ae272f0a319e91a4d849cfc232c90bfd5cdd0b349b9490f89b85c0 WatchSource:0}: Error finding container 0bd70847d3ae272f0a319e91a4d849cfc232c90bfd5cdd0b349b9490f89b85c0: Status 404 returned error can't find the container with id 0bd70847d3ae272f0a319e91a4d849cfc232c90bfd5cdd0b349b9490f89b85c0
Sep 16 18:42:18.428711 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:18.428646    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8" event={"ID":"b8172967-90c2-4159-9b6d-b0838c6703af","Type":"ContainerStarted","Data":"3d8b5fbed2229e843a4a44a13494c9ed1ed4c809486bff7e68ae1e95f5460df2"}
Sep 16 18:42:18.428907 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:18.428721    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8" event={"ID":"b8172967-90c2-4159-9b6d-b0838c6703af","Type":"ContainerStarted","Data":"23aeb69b2b9e64f73f9cc8e3c4db13c289f12ab55d9db999533d07b4ec7088c5"}
Sep 16 18:42:18.428907 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:18.428753    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8" event={"ID":"b8172967-90c2-4159-9b6d-b0838c6703af","Type":"ContainerStarted","Data":"0bd70847d3ae272f0a319e91a4d849cfc232c90bfd5cdd0b349b9490f89b85c0"}
Sep 16 18:42:18.430059 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:18.430027    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-clkzl" event={"ID":"df0ff542-aa25-4096-b1d5-3064c5b308b3","Type":"ContainerStarted","Data":"c94e95e3dde87f2af5eea809495064bd4539326327e9e206d53082e0bb7eccec"}
Sep 16 18:42:19.439071 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:19.439031    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8" event={"ID":"b8172967-90c2-4159-9b6d-b0838c6703af","Type":"ContainerStarted","Data":"f3882242e3d6ee130c89acd4c1400f83500f1df9c65de96dfaf2444220867fe3"}
Sep 16 18:42:19.440815 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:19.440787    2578 generic.go:356] "Generic (PLEG): container finished" podID="df0ff542-aa25-4096-b1d5-3064c5b308b3" containerID="fda781258217e6a13c7639c0f4ad32b1d1db66797c5f732cd7170257be1ef692" exitCode=0
Sep 16 18:42:19.440921 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:19.440853    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-clkzl" event={"ID":"df0ff542-aa25-4096-b1d5-3064c5b308b3","Type":"ContainerDied","Data":"fda781258217e6a13c7639c0f4ad32b1d1db66797c5f732cd7170257be1ef692"}
Sep 16 18:42:19.507044 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:19.506988    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jfq8" podStartSLOduration=1.6461269729999999 podStartE2EDuration="2.50696996s" totalImagesPullingTime="860.842987ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:19.504683708 +0000 UTC m=+167.292247863" watchObservedRunningTime="2026-09-16 18:42:19.50696996 +0000 UTC m=+167.294534115"
Sep 16 18:42:20.447065 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:20.447023    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-clkzl" event={"ID":"df0ff542-aa25-4096-b1d5-3064c5b308b3","Type":"ContainerStarted","Data":"9f820a09d1b518fb118687a3764c6ecda4152e0cf0870a74047d64f2a57c473c"}
Sep 16 18:42:20.447563 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:20.447087    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-clkzl" event={"ID":"df0ff542-aa25-4096-b1d5-3064c5b308b3","Type":"ContainerStarted","Data":"5ae8e67b37014f57c89d476d7405315c807e78ec58a0cb1be008ffd11576f6e8"}
Sep 16 18:42:20.479282 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:20.479232    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-clkzl" podStartSLOduration=2.560419038 podStartE2EDuration="3.479216681s" totalImagesPullingTime="838.304729ms" totalInitContainerRuntime="80.492914ms" isStatefulPod=true podCreationTimestamp="2026-09-16 18:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:20.478083087 +0000 UTC m=+168.265647262" watchObservedRunningTime="2026-09-16 18:42:20.479216681 +0000 UTC m=+168.266780835"
Sep 16 18:42:21.702539 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:21.702497    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:42:22.885838 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:22.885807    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-c6584dd76-xsv84"]
Sep 16 18:42:22.891945 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:22.891927    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:22.896418 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:22.896395    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 16 18:42:22.896655 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:22.896642    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 16 18:42:22.897409 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:22.897384    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 16 18:42:22.897531 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:22.897391    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 16 18:42:22.897608 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:22.897556    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-hqhcq\""
Sep 16 18:42:22.897982 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:22.897966    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 16 18:42:22.905359 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:22.905332    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 16 18:42:22.909042 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:22.909017    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c6584dd76-xsv84"]
Sep 16 18:42:23.075754 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.075718    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk58b\" (UniqueName: \"kubernetes.io/projected/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-kube-api-access-tk58b\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.075754 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.075759    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.075986 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.075786    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-secret-telemeter-client\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.075986 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.075840    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.075986 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.075867    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-telemeter-client-tls\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.075986 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.075894    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-metrics-client-ca\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.075986 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.075925    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-serving-certs-ca-bundle\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.075986 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.075939    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-federate-client-tls\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.177126 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.177048    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-metrics-client-ca\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.177126 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.177099    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-serving-certs-ca-bundle\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.177126 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.177120    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-federate-client-tls\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.177373 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.177151    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk58b\" (UniqueName: \"kubernetes.io/projected/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-kube-api-access-tk58b\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.177373 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.177199    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.177485 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.177405    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-secret-telemeter-client\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.177485 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.177474    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.177591 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.177520    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-telemeter-client-tls\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.177890 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.177866    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-serving-certs-ca-bundle\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.178116 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.178092    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-metrics-client-ca\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.178294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.178274    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.179739 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.179711    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.179877 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.179856    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-secret-telemeter-client\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.180019 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.180002    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-federate-client-tls\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.180064 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.180043    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-telemeter-client-tls\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.187593 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.187575    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk58b\" (UniqueName: \"kubernetes.io/projected/12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc-kube-api-access-tk58b\") pod \"telemeter-client-c6584dd76-xsv84\" (UID: \"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc\") " pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.200941 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.200915    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84"
Sep 16 18:42:23.482996 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:23.482972    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c6584dd76-xsv84"]
Sep 16 18:42:23.484995 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:42:23.484965    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12dd3f0e_d348_4a70_a2b9_eb9922ee0dcc.slice/crio-17b1fb4ea97bb9dbc2b41b19c467f87f1b508a04f16a35747b053c3d496bde52 WatchSource:0}: Error finding container 17b1fb4ea97bb9dbc2b41b19c467f87f1b508a04f16a35747b053c3d496bde52: Status 404 returned error can't find the container with id 17b1fb4ea97bb9dbc2b41b19c467f87f1b508a04f16a35747b053c3d496bde52
Sep 16 18:42:24.464569 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:24.464525    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84" event={"ID":"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc","Type":"ContainerStarted","Data":"17b1fb4ea97bb9dbc2b41b19c467f87f1b508a04f16a35747b053c3d496bde52"}
Sep 16 18:42:24.923930 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:24.923885    2578 patch_prober.go:28] interesting pod/image-registry-68974dbd59-lgnkg container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 16 18:42:24.924233 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:24.923965    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-68974dbd59-lgnkg" podUID="9cebc0d4-b215-494b-b236-32640184278b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:42:25.470133 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:25.470036    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84" event={"ID":"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc","Type":"ContainerStarted","Data":"8074189574f84e300b950132cd77fc1ae8b6268ef8d952d9cbc05e2d038c93c8"}
Sep 16 18:42:26.373629 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:26.373602    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-68974dbd59-lgnkg"
Sep 16 18:42:26.424519 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:26.424489    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-crt2q"
Sep 16 18:42:26.476287 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:26.476208    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84" event={"ID":"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc","Type":"ContainerStarted","Data":"706c7a560c21438212e84840b9927362ca019274c8b6896369cc89896908fa5a"}
Sep 16 18:42:26.476287 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:26.476263    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84" event={"ID":"12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc","Type":"ContainerStarted","Data":"d117b1e849e7888681613a6c9385450941009a75240bbeb7b4ebb44ffa2a5c6a"}
Sep 16 18:42:26.624195 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:26.624113    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-c6584dd76-xsv84" podStartSLOduration=1.973822056 podStartE2EDuration="4.624095387s" totalImagesPullingTime="2.650273331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:26.6237709 +0000 UTC m=+174.411335074" watchObservedRunningTime="2026-09-16 18:42:26.624095387 +0000 UTC m=+174.411659542"
Sep 16 18:42:28.703670 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:28.703631    2578 scope.go:117] "RemoveContainer" containerID="7d9930c95a2d36cfe3ee8a39a7e546c5d23417e64fea20d63288f8bc4a62acaf"
Sep 16 18:42:29.490014 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:29.489983    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 18:42:29.490198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:29.490058    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f" event={"ID":"845d9b0a-5790-4a60-894b-f6b9adce026d","Type":"ContainerStarted","Data":"cb6bb7a81b35708a35086f3b87ab93c16014210d066b1c60c2d35f3ba4f32aab"}
Sep 16 18:42:29.490405 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:29.490378    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:42:29.496124 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:29.496100    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f"
Sep 16 18:42:29.569355 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:29.569300    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-ttc6f" podStartSLOduration=55.671872795 podStartE2EDuration="57.569281557s" totalImagesPullingTime="1.897408762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:29.566044987 +0000 UTC m=+177.353609143" watchObservedRunningTime="2026-09-16 18:42:29.569281557 +0000 UTC m=+177.356845712"
Sep 16 18:42:29.606108 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:29.606075    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-mthnh"]
Sep 16 18:42:29.609665 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:29.609644    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-mthnh"
Sep 16 18:42:29.616953 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:29.616928    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 16 18:42:29.617069 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:29.616933    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 16 18:42:29.617771 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:29.617754    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-bsmtc\""
Sep 16 18:42:29.630499 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:29.630470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89kqk\" (UniqueName: \"kubernetes.io/projected/ee22aa60-e093-4d9b-87bb-46d5e6fdd649-kube-api-access-89kqk\") pod \"downloads-76f99584f4-mthnh\" (UID: \"ee22aa60-e093-4d9b-87bb-46d5e6fdd649\") " pod="openshift-console/downloads-76f99584f4-mthnh"
Sep 16 18:42:29.647353 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:29.647325    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-mthnh"]
Sep 16 18:42:29.731562 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:29.731527    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89kqk\" (UniqueName: \"kubernetes.io/projected/ee22aa60-e093-4d9b-87bb-46d5e6fdd649-kube-api-access-89kqk\") pod \"downloads-76f99584f4-mthnh\" (UID: \"ee22aa60-e093-4d9b-87bb-46d5e6fdd649\") " pod="openshift-console/downloads-76f99584f4-mthnh"
Sep 16 18:42:29.753016 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:29.752943    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89kqk\" (UniqueName: \"kubernetes.io/projected/ee22aa60-e093-4d9b-87bb-46d5e6fdd649-kube-api-access-89kqk\") pod \"downloads-76f99584f4-mthnh\" (UID: \"ee22aa60-e093-4d9b-87bb-46d5e6fdd649\") " pod="openshift-console/downloads-76f99584f4-mthnh"
Sep 16 18:42:29.918704 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:29.918666    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-mthnh"
Sep 16 18:42:30.201478 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:30.201442    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-mthnh"]
Sep 16 18:42:30.204314 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:42:30.204284    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee22aa60_e093_4d9b_87bb_46d5e6fdd649.slice/crio-76bc99a9bc32555067c7ef45ff91995885f03cc3252a4be80f668ea14147dede WatchSource:0}: Error finding container 76bc99a9bc32555067c7ef45ff91995885f03cc3252a4be80f668ea14147dede: Status 404 returned error can't find the container with id 76bc99a9bc32555067c7ef45ff91995885f03cc3252a4be80f668ea14147dede
Sep 16 18:42:30.495262 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:30.495153    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-mthnh" event={"ID":"ee22aa60-e093-4d9b-87bb-46d5e6fdd649","Type":"ContainerStarted","Data":"76bc99a9bc32555067c7ef45ff91995885f03cc3252a4be80f668ea14147dede"}
Sep 16 18:42:46.569121 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:46.569021    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-mthnh" event={"ID":"ee22aa60-e093-4d9b-87bb-46d5e6fdd649","Type":"ContainerStarted","Data":"626d870e42b0efab893761fc75f7e91d07d85ce579de64e22084c2002f950821"}
Sep 16 18:42:46.569662 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:46.569302    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-mthnh"
Sep 16 18:42:46.585685 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:46.585653    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-mthnh"
Sep 16 18:42:46.589825 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:46.589766    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-mthnh" podStartSLOduration=1.572875971 podStartE2EDuration="17.589734496s" totalImagesPullingTime="16.016858525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:46.58896345 +0000 UTC m=+194.376527631" watchObservedRunningTime="2026-09-16 18:42:46.589734496 +0000 UTC m=+194.377298645"
Sep 16 18:42:48.272032 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.271993    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85f7674574-b5t6j"]
Sep 16 18:42:48.298931 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.298895    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85f7674574-b5t6j"]
Sep 16 18:42:48.299117 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.299042    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.305890 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.305860    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 16 18:42:48.306251 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.306231    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 16 18:42:48.306251 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.306242    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 16 18:42:48.307220 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.307201    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 16 18:42:48.307320 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.307207    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-rwxg4\""
Sep 16 18:42:48.307567 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.307549    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 16 18:42:48.314445 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.314421    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 16 18:42:48.406042 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.406004    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-trusted-ca-bundle\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.406241 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.406063    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-console-config\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.406241 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.406098    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f07eca6-705b-434e-99d1-012dae3acf7a-console-serving-cert\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.406241 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.406133    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggnkh\" (UniqueName: \"kubernetes.io/projected/1f07eca6-705b-434e-99d1-012dae3acf7a-kube-api-access-ggnkh\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.406241 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.406215    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-service-ca\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.406429 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.406270    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-oauth-serving-cert\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.406429 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.406355    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f07eca6-705b-434e-99d1-012dae3acf7a-console-oauth-config\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.507588 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.507543    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f07eca6-705b-434e-99d1-012dae3acf7a-console-oauth-config\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.507771 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.507604    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-trusted-ca-bundle\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.507771 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.507650    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-console-config\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.507771 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.507687    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f07eca6-705b-434e-99d1-012dae3acf7a-console-serving-cert\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.507771 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.507724    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggnkh\" (UniqueName: \"kubernetes.io/projected/1f07eca6-705b-434e-99d1-012dae3acf7a-kube-api-access-ggnkh\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.507771 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.507760    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-service-ca\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.508046 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.507795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-oauth-serving-cert\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.508646 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.508620    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-trusted-ca-bundle\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.508894 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.508874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-console-config\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.508959 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.508899    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-service-ca\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.508959 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.508925    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-oauth-serving-cert\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.510501 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.510478    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f07eca6-705b-434e-99d1-012dae3acf7a-console-oauth-config\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.510760 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.510739    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f07eca6-705b-434e-99d1-012dae3acf7a-console-serving-cert\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.519215 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.519191    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggnkh\" (UniqueName: \"kubernetes.io/projected/1f07eca6-705b-434e-99d1-012dae3acf7a-kube-api-access-ggnkh\") pod \"console-85f7674574-b5t6j\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") " pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.611530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.611490    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:48.928371 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:48.928270    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85f7674574-b5t6j"]
Sep 16 18:42:48.930954 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:42:48.930906    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f07eca6_705b_434e_99d1_012dae3acf7a.slice/crio-805e17d19a97fe483c7b91797d46ad995e40c4213f33e9ac729dfa482d6f4a9f WatchSource:0}: Error finding container 805e17d19a97fe483c7b91797d46ad995e40c4213f33e9ac729dfa482d6f4a9f: Status 404 returned error can't find the container with id 805e17d19a97fe483c7b91797d46ad995e40c4213f33e9ac729dfa482d6f4a9f
Sep 16 18:42:49.585579 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:49.585528    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85f7674574-b5t6j" event={"ID":"1f07eca6-705b-434e-99d1-012dae3acf7a","Type":"ContainerStarted","Data":"805e17d19a97fe483c7b91797d46ad995e40c4213f33e9ac729dfa482d6f4a9f"}
Sep 16 18:42:53.607163 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:53.607120    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85f7674574-b5t6j" event={"ID":"1f07eca6-705b-434e-99d1-012dae3acf7a","Type":"ContainerStarted","Data":"f4a64c46c102db4a14260a56adf628ddb42fc78dd6156cdfefd3df0496bdb573"}
Sep 16 18:42:53.626950 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:53.626895    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85f7674574-b5t6j" podStartSLOduration=1.995215928 podStartE2EDuration="5.626877771s" totalImagesPullingTime="3.631661843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:53.626090143 +0000 UTC m=+201.413654316" watchObservedRunningTime="2026-09-16 18:42:53.626877771 +0000 UTC m=+201.414441926"
Sep 16 18:42:57.627138 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:57.627102    2578 generic.go:356] "Generic (PLEG): container finished" podID="306e53f3-37de-49c6-a491-f9faf67d7f2a" containerID="5f0a9273991b6fab4303c8cbddd40576064a779b4ecaf5ff69a90635df565de8" exitCode=0
Sep 16 18:42:57.627609 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:57.627162    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-vhrf8" event={"ID":"306e53f3-37de-49c6-a491-f9faf67d7f2a","Type":"ContainerDied","Data":"5f0a9273991b6fab4303c8cbddd40576064a779b4ecaf5ff69a90635df565de8"}
Sep 16 18:42:57.627609 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:57.627548    2578 scope.go:117] "RemoveContainer" containerID="5f0a9273991b6fab4303c8cbddd40576064a779b4ecaf5ff69a90635df565de8"
Sep 16 18:42:58.612015 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:58.611979    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:58.612199 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:58.612047    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:58.616912 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:58.616885    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:42:58.618499 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:58.618474    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-h6s5l_f21dc980-1bae-414a-be6b-7145bcc7c7f5/serve-healthcheck-canary/0.log"
Sep 16 18:42:58.632674 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:58.632642    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-vhrf8" event={"ID":"306e53f3-37de-49c6-a491-f9faf67d7f2a","Type":"ContainerStarted","Data":"f84e26246a7433fe0fa28c2f0f4226f9110df31c1cd1a61d3ec87ab22445edcd"}
Sep 16 18:42:58.637267 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:42:58.637224    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:43:43.485266 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:43:43.485226    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs\") pod \"network-metrics-daemon-282m9\" (UID: \"87684818-77c6-45bc-bee8-61a206348549\") " pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:43:43.487557 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:43:43.487533    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87684818-77c6-45bc-bee8-61a206348549-metrics-certs\") pod \"network-metrics-daemon-282m9\" (UID: \"87684818-77c6-45bc-bee8-61a206348549\") " pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:43:43.605637 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:43:43.605603    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l6kxj\""
Sep 16 18:43:43.613581 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:43:43.613562    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-282m9"
Sep 16 18:43:43.884825 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:43:43.884775    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-282m9"]
Sep 16 18:43:43.887156 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:43:43.887121    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87684818_77c6_45bc_bee8_61a206348549.slice/crio-ecc0e7eddb117205c94a09b527a59c9b8e3cecd23ef48220f4c7797e42e6ecf1 WatchSource:0}: Error finding container ecc0e7eddb117205c94a09b527a59c9b8e3cecd23ef48220f4c7797e42e6ecf1: Status 404 returned error can't find the container with id ecc0e7eddb117205c94a09b527a59c9b8e3cecd23ef48220f4c7797e42e6ecf1
Sep 16 18:43:44.817503 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:43:44.817454    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-282m9" event={"ID":"87684818-77c6-45bc-bee8-61a206348549","Type":"ContainerStarted","Data":"ecc0e7eddb117205c94a09b527a59c9b8e3cecd23ef48220f4c7797e42e6ecf1"}
Sep 16 18:43:45.823491 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:43:45.823452    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-282m9" event={"ID":"87684818-77c6-45bc-bee8-61a206348549","Type":"ContainerStarted","Data":"bf23d735406a08a186c922f86dd4c80551949b8329b69b1c5e453fdbb4db810d"}
Sep 16 18:43:45.823877 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:43:45.823537    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-282m9" event={"ID":"87684818-77c6-45bc-bee8-61a206348549","Type":"ContainerStarted","Data":"f829ad778fd8b1fe061f617a7e3f515b5a7c161f1cd780fc9a82b6ec8f5b6b59"}
Sep 16 18:43:45.843494 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:43:45.843440    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-282m9" podStartSLOduration=251.953399372 podStartE2EDuration="4m12.843422293s" totalImagesPullingTime="890.022921ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:43:45.841332099 +0000 UTC m=+253.628896255" watchObservedRunningTime="2026-09-16 18:43:45.843422293 +0000 UTC m=+253.630986448"
Sep 16 18:44:32.613056 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:44:32.613026    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 18:44:32.619212 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:44:32.619160    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 18:44:32.623668 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:44:32.623645    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 18:44:32.626610 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:44:32.626585    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 18:44:32.628779 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:44:32.628761    2578 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 18:48:53.883367 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:53.883333    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5747b7df4-z87km"]
Sep 16 18:48:53.885466 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:53.885446    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:53.899760 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:53.899735    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5747b7df4-z87km"]
Sep 16 18:48:54.002930 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.002900    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-service-ca\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.003116 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.002939    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-trusted-ca-bundle\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.003116 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.002975    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-console-config\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.003116 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.002995    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-oauth-serving-cert\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.003116 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.003013    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17943503-a564-4ab8-b255-1173fe3f15ef-console-oauth-config\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.003116 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.003031    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh44z\" (UniqueName: \"kubernetes.io/projected/17943503-a564-4ab8-b255-1173fe3f15ef-kube-api-access-qh44z\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.003116 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.003077    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17943503-a564-4ab8-b255-1173fe3f15ef-console-serving-cert\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.104126 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.104095    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-trusted-ca-bundle\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.104325 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.104134    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-console-config\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.104325 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.104150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-oauth-serving-cert\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.104325 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.104278    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17943503-a564-4ab8-b255-1173fe3f15ef-console-oauth-config\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.104325 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.104324    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh44z\" (UniqueName: \"kubernetes.io/projected/17943503-a564-4ab8-b255-1173fe3f15ef-kube-api-access-qh44z\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.104539 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.104378    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17943503-a564-4ab8-b255-1173fe3f15ef-console-serving-cert\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.104539 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.104479    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-service-ca\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.104910 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.104886    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-oauth-serving-cert\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.104910 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.104899    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-console-config\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.105085 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.105045    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-trusted-ca-bundle\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.105137 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.105080    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-service-ca\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.106785 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.106762    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17943503-a564-4ab8-b255-1173fe3f15ef-console-serving-cert\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.106868 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.106787    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17943503-a564-4ab8-b255-1173fe3f15ef-console-oauth-config\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.115862 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.115842    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh44z\" (UniqueName: \"kubernetes.io/projected/17943503-a564-4ab8-b255-1173fe3f15ef-kube-api-access-qh44z\") pod \"console-5747b7df4-z87km\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") " pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.196049 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.195947    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:48:54.477245 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:54.477149    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5747b7df4-z87km"]
Sep 16 18:48:54.480115 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:48:54.480088    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17943503_a564_4ab8_b255_1173fe3f15ef.slice/crio-b0673aadd3375e0dbff82dddb0b61093bf33b3dcc050030f2edc025018d946f4 WatchSource:0}: Error finding container b0673aadd3375e0dbff82dddb0b61093bf33b3dcc050030f2edc025018d946f4: Status 404 returned error can't find the container with id b0673aadd3375e0dbff82dddb0b61093bf33b3dcc050030f2edc025018d946f4
Sep 16 18:48:55.009254 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:55.009217    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5747b7df4-z87km" event={"ID":"17943503-a564-4ab8-b255-1173fe3f15ef","Type":"ContainerStarted","Data":"79af1c1f1983453bfb95741d7d8e2b43a10d8f53f648bc18fef2e9e7b445b885"}
Sep 16 18:48:55.009672 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:55.009272    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5747b7df4-z87km" event={"ID":"17943503-a564-4ab8-b255-1173fe3f15ef","Type":"ContainerStarted","Data":"b0673aadd3375e0dbff82dddb0b61093bf33b3dcc050030f2edc025018d946f4"}
Sep 16 18:48:55.032987 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:48:55.032938    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5747b7df4-z87km" podStartSLOduration=2.032920553 podStartE2EDuration="2.032920553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:48:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:48:55.031782862 +0000 UTC m=+562.819347047" watchObservedRunningTime="2026-09-16 18:48:55.032920553 +0000 UTC m=+562.820484709"
Sep 16 18:49:04.196558 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:04.196514    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:49:04.196987 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:04.196581    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:49:04.201426 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:04.201394    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:49:05.052036 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:05.052006    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:49:05.109832 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:05.109802    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85f7674574-b5t6j"]
Sep 16 18:49:30.130082 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.130031    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85f7674574-b5t6j" podUID="1f07eca6-705b-434e-99d1-012dae3acf7a" containerName="console" containerID="cri-o://f4a64c46c102db4a14260a56adf628ddb42fc78dd6156cdfefd3df0496bdb573" gracePeriod=15
Sep 16 18:49:30.374035 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.374010    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:49:30.407696 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.407620    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggnkh\" (UniqueName: \"kubernetes.io/projected/1f07eca6-705b-434e-99d1-012dae3acf7a-kube-api-access-ggnkh\") pod \"1f07eca6-705b-434e-99d1-012dae3acf7a\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") "
Sep 16 18:49:30.407696 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.407671    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-console-config\") pod \"1f07eca6-705b-434e-99d1-012dae3acf7a\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") "
Sep 16 18:49:30.407696 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.407695    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f07eca6-705b-434e-99d1-012dae3acf7a-console-oauth-config\") pod \"1f07eca6-705b-434e-99d1-012dae3acf7a\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") "
Sep 16 18:49:30.407931 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.407739    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f07eca6-705b-434e-99d1-012dae3acf7a-console-serving-cert\") pod \"1f07eca6-705b-434e-99d1-012dae3acf7a\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") "
Sep 16 18:49:30.407931 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.407764    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-service-ca\") pod \"1f07eca6-705b-434e-99d1-012dae3acf7a\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") "
Sep 16 18:49:30.407931 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.407798    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-trusted-ca-bundle\") pod \"1f07eca6-705b-434e-99d1-012dae3acf7a\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") "
Sep 16 18:49:30.407931 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.407851    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-oauth-serving-cert\") pod \"1f07eca6-705b-434e-99d1-012dae3acf7a\" (UID: \"1f07eca6-705b-434e-99d1-012dae3acf7a\") "
Sep 16 18:49:30.408198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.408149    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-console-config" (OuterVolumeSpecName: "console-config") pod "1f07eca6-705b-434e-99d1-012dae3acf7a" (UID: "1f07eca6-705b-434e-99d1-012dae3acf7a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:49:30.408430 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.408408    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1f07eca6-705b-434e-99d1-012dae3acf7a" (UID: "1f07eca6-705b-434e-99d1-012dae3acf7a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:49:30.408509 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.408437    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1f07eca6-705b-434e-99d1-012dae3acf7a" (UID: "1f07eca6-705b-434e-99d1-012dae3acf7a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:49:30.408571 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.408522    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-service-ca" (OuterVolumeSpecName: "service-ca") pod "1f07eca6-705b-434e-99d1-012dae3acf7a" (UID: "1f07eca6-705b-434e-99d1-012dae3acf7a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:49:30.410160 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.410128    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f07eca6-705b-434e-99d1-012dae3acf7a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1f07eca6-705b-434e-99d1-012dae3acf7a" (UID: "1f07eca6-705b-434e-99d1-012dae3acf7a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:49:30.410334 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.410222    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f07eca6-705b-434e-99d1-012dae3acf7a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1f07eca6-705b-434e-99d1-012dae3acf7a" (UID: "1f07eca6-705b-434e-99d1-012dae3acf7a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:49:30.410334 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.410325    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f07eca6-705b-434e-99d1-012dae3acf7a-kube-api-access-ggnkh" (OuterVolumeSpecName: "kube-api-access-ggnkh") pod "1f07eca6-705b-434e-99d1-012dae3acf7a" (UID: "1f07eca6-705b-434e-99d1-012dae3acf7a"). InnerVolumeSpecName "kube-api-access-ggnkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:49:30.509062 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.509026    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f07eca6-705b-434e-99d1-012dae3acf7a-console-serving-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:49:30.509062 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.509053    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-service-ca\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:49:30.509062 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.509063    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-trusted-ca-bundle\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:49:30.509062 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.509071    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-oauth-serving-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:49:30.509340 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.509080    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggnkh\" (UniqueName: \"kubernetes.io/projected/1f07eca6-705b-434e-99d1-012dae3acf7a-kube-api-access-ggnkh\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:49:30.509340 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.509088    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f07eca6-705b-434e-99d1-012dae3acf7a-console-config\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:49:30.509340 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:30.509101    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f07eca6-705b-434e-99d1-012dae3acf7a-console-oauth-config\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:49:31.153324 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:31.153289    2578 generic.go:356] "Generic (PLEG): container finished" podID="1f07eca6-705b-434e-99d1-012dae3acf7a" containerID="f4a64c46c102db4a14260a56adf628ddb42fc78dd6156cdfefd3df0496bdb573" exitCode=0
Sep 16 18:49:31.153727 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:31.153335    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85f7674574-b5t6j"
Sep 16 18:49:31.153727 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:31.153372    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85f7674574-b5t6j" event={"ID":"1f07eca6-705b-434e-99d1-012dae3acf7a","Type":"ContainerDied","Data":"f4a64c46c102db4a14260a56adf628ddb42fc78dd6156cdfefd3df0496bdb573"}
Sep 16 18:49:31.153727 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:31.153429    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85f7674574-b5t6j" event={"ID":"1f07eca6-705b-434e-99d1-012dae3acf7a","Type":"ContainerDied","Data":"805e17d19a97fe483c7b91797d46ad995e40c4213f33e9ac729dfa482d6f4a9f"}
Sep 16 18:49:31.153727 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:31.153451    2578 scope.go:117] "RemoveContainer" containerID="f4a64c46c102db4a14260a56adf628ddb42fc78dd6156cdfefd3df0496bdb573"
Sep 16 18:49:31.161707 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:31.161691    2578 scope.go:117] "RemoveContainer" containerID="f4a64c46c102db4a14260a56adf628ddb42fc78dd6156cdfefd3df0496bdb573"
Sep 16 18:49:31.162001 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:49:31.161981    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4a64c46c102db4a14260a56adf628ddb42fc78dd6156cdfefd3df0496bdb573\": container with ID starting with f4a64c46c102db4a14260a56adf628ddb42fc78dd6156cdfefd3df0496bdb573 not found: ID does not exist" containerID="f4a64c46c102db4a14260a56adf628ddb42fc78dd6156cdfefd3df0496bdb573"
Sep 16 18:49:31.162063 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:31.162010    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a64c46c102db4a14260a56adf628ddb42fc78dd6156cdfefd3df0496bdb573"} err="failed to get container status \"f4a64c46c102db4a14260a56adf628ddb42fc78dd6156cdfefd3df0496bdb573\": rpc error: code = NotFound desc = could not find container \"f4a64c46c102db4a14260a56adf628ddb42fc78dd6156cdfefd3df0496bdb573\": container with ID starting with f4a64c46c102db4a14260a56adf628ddb42fc78dd6156cdfefd3df0496bdb573 not found: ID does not exist"
Sep 16 18:49:31.170994 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:31.170955    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85f7674574-b5t6j"]
Sep 16 18:49:31.174694 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:31.174666    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85f7674574-b5t6j"]
Sep 16 18:49:32.644603 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:32.644576    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 18:49:32.645294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:32.645272    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 18:49:32.651017 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:32.650996    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 18:49:32.651969 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:32.651950    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 18:49:32.707969 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:32.707941    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f07eca6-705b-434e-99d1-012dae3acf7a" path="/var/lib/kubelet/pods/1f07eca6-705b-434e-99d1-012dae3acf7a/volumes"
Sep 16 18:49:44.192268 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.192233    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8"]
Sep 16 18:49:44.192668 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.192520    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f07eca6-705b-434e-99d1-012dae3acf7a" containerName="console"
Sep 16 18:49:44.192668 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.192531    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f07eca6-705b-434e-99d1-012dae3acf7a" containerName="console"
Sep 16 18:49:44.192668 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.192594    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f07eca6-705b-434e-99d1-012dae3acf7a" containerName="console"
Sep 16 18:49:44.195567 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.195547    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8"
Sep 16 18:49:44.198648 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.198610    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 18:49:44.199727 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.199702    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5tnch\""
Sep 16 18:49:44.199859 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.199812    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 18:49:44.205352 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.205326    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8"]
Sep 16 18:49:44.316935 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.316891    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfpdx\" (UniqueName: \"kubernetes.io/projected/dda71b9c-67bb-4226-967c-a0f10215766a-kube-api-access-dfpdx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8\" (UID: \"dda71b9c-67bb-4226-967c-a0f10215766a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8"
Sep 16 18:49:44.316935 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.316938    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dda71b9c-67bb-4226-967c-a0f10215766a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8\" (UID: \"dda71b9c-67bb-4226-967c-a0f10215766a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8"
Sep 16 18:49:44.317153 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.317068    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dda71b9c-67bb-4226-967c-a0f10215766a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8\" (UID: \"dda71b9c-67bb-4226-967c-a0f10215766a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8"
Sep 16 18:49:44.417920 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.417876    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dda71b9c-67bb-4226-967c-a0f10215766a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8\" (UID: \"dda71b9c-67bb-4226-967c-a0f10215766a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8"
Sep 16 18:49:44.418114 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.417950    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfpdx\" (UniqueName: \"kubernetes.io/projected/dda71b9c-67bb-4226-967c-a0f10215766a-kube-api-access-dfpdx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8\" (UID: \"dda71b9c-67bb-4226-967c-a0f10215766a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8"
Sep 16 18:49:44.418114 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.417977    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dda71b9c-67bb-4226-967c-a0f10215766a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8\" (UID: \"dda71b9c-67bb-4226-967c-a0f10215766a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8"
Sep 16 18:49:44.418307 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.418285    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dda71b9c-67bb-4226-967c-a0f10215766a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8\" (UID: \"dda71b9c-67bb-4226-967c-a0f10215766a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8"
Sep 16 18:49:44.418375 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.418349    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dda71b9c-67bb-4226-967c-a0f10215766a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8\" (UID: \"dda71b9c-67bb-4226-967c-a0f10215766a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8"
Sep 16 18:49:44.428604 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.428568    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfpdx\" (UniqueName: \"kubernetes.io/projected/dda71b9c-67bb-4226-967c-a0f10215766a-kube-api-access-dfpdx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8\" (UID: \"dda71b9c-67bb-4226-967c-a0f10215766a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8"
Sep 16 18:49:44.506111 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.506023    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8"
Sep 16 18:49:44.784042 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.784018    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8"]
Sep 16 18:49:44.785955 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:49:44.785918    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda71b9c_67bb_4226_967c_a0f10215766a.slice/crio-4657817b3bc0a6d62ca998f9264684a147210c5ebef3d0a0ce22aa8c0073692b WatchSource:0}: Error finding container 4657817b3bc0a6d62ca998f9264684a147210c5ebef3d0a0ce22aa8c0073692b: Status 404 returned error can't find the container with id 4657817b3bc0a6d62ca998f9264684a147210c5ebef3d0a0ce22aa8c0073692b
Sep 16 18:49:44.787844 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:44.787828    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:49:45.209069 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:45.209034    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8" event={"ID":"dda71b9c-67bb-4226-967c-a0f10215766a","Type":"ContainerStarted","Data":"4657817b3bc0a6d62ca998f9264684a147210c5ebef3d0a0ce22aa8c0073692b"}
Sep 16 18:49:51.236970 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:51.236937    2578 generic.go:356] "Generic (PLEG): container finished" podID="dda71b9c-67bb-4226-967c-a0f10215766a" containerID="09080a3e9d812415278f6ee468beb147c91c98f4d07874b860aa869121215f66" exitCode=0
Sep 16 18:49:51.237383 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:51.236987    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8" event={"ID":"dda71b9c-67bb-4226-967c-a0f10215766a","Type":"ContainerDied","Data":"09080a3e9d812415278f6ee468beb147c91c98f4d07874b860aa869121215f66"}
Sep 16 18:49:54.257046 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:54.256997    2578 generic.go:356] "Generic (PLEG): container finished" podID="dda71b9c-67bb-4226-967c-a0f10215766a" containerID="28e29af5490e148910a98f1966d4ac038c953db0786222897809e0bc2ac3de5c" exitCode=0
Sep 16 18:49:54.257508 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:49:54.257069    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8" event={"ID":"dda71b9c-67bb-4226-967c-a0f10215766a","Type":"ContainerDied","Data":"28e29af5490e148910a98f1966d4ac038c953db0786222897809e0bc2ac3de5c"}
Sep 16 18:50:02.293404 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:02.293371    2578 generic.go:356] "Generic (PLEG): container finished" podID="dda71b9c-67bb-4226-967c-a0f10215766a" containerID="fa8ea298c9a9205082a3f7ed891f937a2ad2b7b830bfa0e17925df6ea2d930c8" exitCode=0
Sep 16 18:50:02.293836 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:02.293452    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8" event={"ID":"dda71b9c-67bb-4226-967c-a0f10215766a","Type":"ContainerDied","Data":"fa8ea298c9a9205082a3f7ed891f937a2ad2b7b830bfa0e17925df6ea2d930c8"}
Sep 16 18:50:03.429995 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:03.429968    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8"
Sep 16 18:50:03.586956 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:03.586861    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dda71b9c-67bb-4226-967c-a0f10215766a-bundle\") pod \"dda71b9c-67bb-4226-967c-a0f10215766a\" (UID: \"dda71b9c-67bb-4226-967c-a0f10215766a\") "
Sep 16 18:50:03.586956 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:03.586950    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dda71b9c-67bb-4226-967c-a0f10215766a-util\") pod \"dda71b9c-67bb-4226-967c-a0f10215766a\" (UID: \"dda71b9c-67bb-4226-967c-a0f10215766a\") "
Sep 16 18:50:03.587133 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:03.587004    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfpdx\" (UniqueName: \"kubernetes.io/projected/dda71b9c-67bb-4226-967c-a0f10215766a-kube-api-access-dfpdx\") pod \"dda71b9c-67bb-4226-967c-a0f10215766a\" (UID: \"dda71b9c-67bb-4226-967c-a0f10215766a\") "
Sep 16 18:50:03.587610 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:03.587586    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dda71b9c-67bb-4226-967c-a0f10215766a-bundle" (OuterVolumeSpecName: "bundle") pod "dda71b9c-67bb-4226-967c-a0f10215766a" (UID: "dda71b9c-67bb-4226-967c-a0f10215766a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:50:03.589212 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:03.589187    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dda71b9c-67bb-4226-967c-a0f10215766a-kube-api-access-dfpdx" (OuterVolumeSpecName: "kube-api-access-dfpdx") pod "dda71b9c-67bb-4226-967c-a0f10215766a" (UID: "dda71b9c-67bb-4226-967c-a0f10215766a"). InnerVolumeSpecName "kube-api-access-dfpdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:50:03.590821 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:03.590797    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dda71b9c-67bb-4226-967c-a0f10215766a-util" (OuterVolumeSpecName: "util") pod "dda71b9c-67bb-4226-967c-a0f10215766a" (UID: "dda71b9c-67bb-4226-967c-a0f10215766a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:50:03.688312 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:03.688277    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dda71b9c-67bb-4226-967c-a0f10215766a-util\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:50:03.688312 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:03.688306    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfpdx\" (UniqueName: \"kubernetes.io/projected/dda71b9c-67bb-4226-967c-a0f10215766a-kube-api-access-dfpdx\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:50:03.688312 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:03.688318    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dda71b9c-67bb-4226-967c-a0f10215766a-bundle\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:50:04.303536 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:04.303495    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8" event={"ID":"dda71b9c-67bb-4226-967c-a0f10215766a","Type":"ContainerDied","Data":"4657817b3bc0a6d62ca998f9264684a147210c5ebef3d0a0ce22aa8c0073692b"}
Sep 16 18:50:04.303743 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:04.303544    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gf4q8"
Sep 16 18:50:04.303743 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:04.303555    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4657817b3bc0a6d62ca998f9264684a147210c5ebef3d0a0ce22aa8c0073692b"
Sep 16 18:50:12.265634 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.265599    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ktvt6"]
Sep 16 18:50:12.287285 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.265916    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda71b9c-67bb-4226-967c-a0f10215766a" containerName="pull"
Sep 16 18:50:12.287285 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.265927    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda71b9c-67bb-4226-967c-a0f10215766a" containerName="pull"
Sep 16 18:50:12.287285 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.265938    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda71b9c-67bb-4226-967c-a0f10215766a" containerName="extract"
Sep 16 18:50:12.287285 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.265944    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda71b9c-67bb-4226-967c-a0f10215766a" containerName="extract"
Sep 16 18:50:12.287285 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.265963    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda71b9c-67bb-4226-967c-a0f10215766a" containerName="util"
Sep 16 18:50:12.287285 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.265969    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda71b9c-67bb-4226-967c-a0f10215766a" containerName="util"
Sep 16 18:50:12.287285 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.266018    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda71b9c-67bb-4226-967c-a0f10215766a" containerName="extract"
Sep 16 18:50:12.297305 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.297280    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ktvt6"]
Sep 16 18:50:12.297446 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.297372    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ktvt6"
Sep 16 18:50:12.301257 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.301228    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 16 18:50:12.301375 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.301276    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-659bp\""
Sep 16 18:50:12.301375 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.301282    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:50:12.468359 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.468316    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fde9b63-2367-4c6f-8b48-0cf38a082fcd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ktvt6\" (UID: \"8fde9b63-2367-4c6f-8b48-0cf38a082fcd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ktvt6"
Sep 16 18:50:12.468551 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.468380    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x94h\" (UniqueName: \"kubernetes.io/projected/8fde9b63-2367-4c6f-8b48-0cf38a082fcd-kube-api-access-2x94h\") pod \"cert-manager-operator-controller-manager-55c444bf86-ktvt6\" (UID: \"8fde9b63-2367-4c6f-8b48-0cf38a082fcd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ktvt6"
Sep 16 18:50:12.569132 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.569047    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fde9b63-2367-4c6f-8b48-0cf38a082fcd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ktvt6\" (UID: \"8fde9b63-2367-4c6f-8b48-0cf38a082fcd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ktvt6"
Sep 16 18:50:12.569132 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.569114    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x94h\" (UniqueName: \"kubernetes.io/projected/8fde9b63-2367-4c6f-8b48-0cf38a082fcd-kube-api-access-2x94h\") pod \"cert-manager-operator-controller-manager-55c444bf86-ktvt6\" (UID: \"8fde9b63-2367-4c6f-8b48-0cf38a082fcd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ktvt6"
Sep 16 18:50:12.569500 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.569481    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fde9b63-2367-4c6f-8b48-0cf38a082fcd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ktvt6\" (UID: \"8fde9b63-2367-4c6f-8b48-0cf38a082fcd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ktvt6"
Sep 16 18:50:12.578607 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.578574    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x94h\" (UniqueName: \"kubernetes.io/projected/8fde9b63-2367-4c6f-8b48-0cf38a082fcd-kube-api-access-2x94h\") pod \"cert-manager-operator-controller-manager-55c444bf86-ktvt6\" (UID: \"8fde9b63-2367-4c6f-8b48-0cf38a082fcd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ktvt6"
Sep 16 18:50:12.606266 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.606228    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ktvt6"
Sep 16 18:50:12.904554 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:12.904516    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ktvt6"]
Sep 16 18:50:12.905653 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:50:12.905623    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fde9b63_2367_4c6f_8b48_0cf38a082fcd.slice/crio-2d6790725441d2c1a3bab9fd3406c5ddb19cbfcacd03c9b64de3e22492229eaf WatchSource:0}: Error finding container 2d6790725441d2c1a3bab9fd3406c5ddb19cbfcacd03c9b64de3e22492229eaf: Status 404 returned error can't find the container with id 2d6790725441d2c1a3bab9fd3406c5ddb19cbfcacd03c9b64de3e22492229eaf
Sep 16 18:50:13.341420 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:13.341321    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ktvt6" event={"ID":"8fde9b63-2367-4c6f-8b48-0cf38a082fcd","Type":"ContainerStarted","Data":"2d6790725441d2c1a3bab9fd3406c5ddb19cbfcacd03c9b64de3e22492229eaf"}
Sep 16 18:50:15.352839 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:15.352803    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ktvt6" event={"ID":"8fde9b63-2367-4c6f-8b48-0cf38a082fcd","Type":"ContainerStarted","Data":"6cc83a91aed4b64ae10a794acd090fc606e95c8ccf1112b96496ee565737cf64"}
Sep 16 18:50:15.377643 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:15.377586    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ktvt6" podStartSLOduration=1.467542614 podStartE2EDuration="3.377567718s" totalImagesPullingTime="1.910025104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:50:15.375419127 +0000 UTC m=+643.162983282" watchObservedRunningTime="2026-09-16 18:50:15.377567718 +0000 UTC m=+643.165131872"
Sep 16 18:50:16.554210 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:16.554160    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9"]
Sep 16 18:50:16.557028 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:16.557012    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9"
Sep 16 18:50:16.559924 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:16.559900    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 18:50:16.560407 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:16.560391    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 18:50:16.562108 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:16.562093    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5tnch\""
Sep 16 18:50:16.605932 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:16.605893    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gz9n\" (UniqueName: \"kubernetes.io/projected/303978d0-6046-43f1-8ba0-5f180332e46e-kube-api-access-4gz9n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9\" (UID: \"303978d0-6046-43f1-8ba0-5f180332e46e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9"
Sep 16 18:50:16.606132 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:16.605970    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/303978d0-6046-43f1-8ba0-5f180332e46e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9\" (UID: \"303978d0-6046-43f1-8ba0-5f180332e46e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9"
Sep 16 18:50:16.606132 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:16.606009    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/303978d0-6046-43f1-8ba0-5f180332e46e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9\" (UID: \"303978d0-6046-43f1-8ba0-5f180332e46e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9"
Sep 16 18:50:16.619468 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:16.619437    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9"]
Sep 16 18:50:16.706416 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:16.706368    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/303978d0-6046-43f1-8ba0-5f180332e46e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9\" (UID: \"303978d0-6046-43f1-8ba0-5f180332e46e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9"
Sep 16 18:50:16.706609 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:16.706442    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/303978d0-6046-43f1-8ba0-5f180332e46e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9\" (UID: \"303978d0-6046-43f1-8ba0-5f180332e46e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9"
Sep 16 18:50:16.706609 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:16.706477    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gz9n\" (UniqueName: \"kubernetes.io/projected/303978d0-6046-43f1-8ba0-5f180332e46e-kube-api-access-4gz9n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9\" (UID: \"303978d0-6046-43f1-8ba0-5f180332e46e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9"
Sep 16 18:50:16.706857 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:16.706831    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/303978d0-6046-43f1-8ba0-5f180332e46e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9\" (UID: \"303978d0-6046-43f1-8ba0-5f180332e46e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9"
Sep 16 18:50:16.706916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:16.706885    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/303978d0-6046-43f1-8ba0-5f180332e46e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9\" (UID: \"303978d0-6046-43f1-8ba0-5f180332e46e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9"
Sep 16 18:50:16.714739 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:16.714716    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gz9n\" (UniqueName: \"kubernetes.io/projected/303978d0-6046-43f1-8ba0-5f180332e46e-kube-api-access-4gz9n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9\" (UID: \"303978d0-6046-43f1-8ba0-5f180332e46e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9"
Sep 16 18:50:16.867030 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:16.867000    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9"
Sep 16 18:50:17.144041 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:17.144014    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9"]
Sep 16 18:50:17.145793 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:50:17.145751    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod303978d0_6046_43f1_8ba0_5f180332e46e.slice/crio-171287d4e2656bcbfb7f00937ca11a5e430dd494b896cfdb2016227b5784c381 WatchSource:0}: Error finding container 171287d4e2656bcbfb7f00937ca11a5e430dd494b896cfdb2016227b5784c381: Status 404 returned error can't find the container with id 171287d4e2656bcbfb7f00937ca11a5e430dd494b896cfdb2016227b5784c381
Sep 16 18:50:17.363793 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:17.363759    2578 generic.go:356] "Generic (PLEG): container finished" podID="303978d0-6046-43f1-8ba0-5f180332e46e" containerID="298c6715e61c560b37559a7e41fcad0c3c45594a5bf33a6c0ded6cd96cf147b4" exitCode=0
Sep 16 18:50:17.363940 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:17.363830    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9" event={"ID":"303978d0-6046-43f1-8ba0-5f180332e46e","Type":"ContainerDied","Data":"298c6715e61c560b37559a7e41fcad0c3c45594a5bf33a6c0ded6cd96cf147b4"}
Sep 16 18:50:17.363940 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:17.363866    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9" event={"ID":"303978d0-6046-43f1-8ba0-5f180332e46e","Type":"ContainerStarted","Data":"171287d4e2656bcbfb7f00937ca11a5e430dd494b896cfdb2016227b5784c381"}
Sep 16 18:50:19.417267 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:19.417235    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cjlxd"]
Sep 16 18:50:19.419371 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:19.419353    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cjlxd"
Sep 16 18:50:19.421757 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:19.421723    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 16 18:50:19.421878 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:19.421848    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 16 18:50:19.422644 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:19.422618    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-22nj8\""
Sep 16 18:50:19.428501 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:19.428478    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ded809c0-38fa-4a8f-aae0-ac11509732a0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cjlxd\" (UID: \"ded809c0-38fa-4a8f-aae0-ac11509732a0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cjlxd"
Sep 16 18:50:19.428594 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:19.428517    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5d7r\" (UniqueName: \"kubernetes.io/projected/ded809c0-38fa-4a8f-aae0-ac11509732a0-kube-api-access-b5d7r\") pod \"cert-manager-cainjector-559cdd8c87-cjlxd\" (UID: \"ded809c0-38fa-4a8f-aae0-ac11509732a0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cjlxd"
Sep 16 18:50:19.431686 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:19.431656    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cjlxd"]
Sep 16 18:50:19.529434 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:19.529394    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ded809c0-38fa-4a8f-aae0-ac11509732a0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cjlxd\" (UID: \"ded809c0-38fa-4a8f-aae0-ac11509732a0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cjlxd"
Sep 16 18:50:19.529612 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:19.529451    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5d7r\" (UniqueName: \"kubernetes.io/projected/ded809c0-38fa-4a8f-aae0-ac11509732a0-kube-api-access-b5d7r\") pod \"cert-manager-cainjector-559cdd8c87-cjlxd\" (UID: \"ded809c0-38fa-4a8f-aae0-ac11509732a0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cjlxd"
Sep 16 18:50:19.537598 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:19.537568    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ded809c0-38fa-4a8f-aae0-ac11509732a0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-cjlxd\" (UID: \"ded809c0-38fa-4a8f-aae0-ac11509732a0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cjlxd"
Sep 16 18:50:19.537729 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:19.537715    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5d7r\" (UniqueName: \"kubernetes.io/projected/ded809c0-38fa-4a8f-aae0-ac11509732a0-kube-api-access-b5d7r\") pod \"cert-manager-cainjector-559cdd8c87-cjlxd\" (UID: \"ded809c0-38fa-4a8f-aae0-ac11509732a0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-cjlxd"
Sep 16 18:50:19.735196 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:19.735145    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cjlxd"
Sep 16 18:50:20.009258 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:20.009230    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-cjlxd"]
Sep 16 18:50:20.011350 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:50:20.011325    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podded809c0_38fa_4a8f_aae0_ac11509732a0.slice/crio-52fe9e8100660a414e75ed0a7a32bc07f9033c86b9dd1116e2df632a631c8940 WatchSource:0}: Error finding container 52fe9e8100660a414e75ed0a7a32bc07f9033c86b9dd1116e2df632a631c8940: Status 404 returned error can't find the container with id 52fe9e8100660a414e75ed0a7a32bc07f9033c86b9dd1116e2df632a631c8940
Sep 16 18:50:20.379525 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:20.379483    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cjlxd" event={"ID":"ded809c0-38fa-4a8f-aae0-ac11509732a0","Type":"ContainerStarted","Data":"52fe9e8100660a414e75ed0a7a32bc07f9033c86b9dd1116e2df632a631c8940"}
Sep 16 18:50:20.381161 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:20.381139    2578 generic.go:356] "Generic (PLEG): container finished" podID="303978d0-6046-43f1-8ba0-5f180332e46e" containerID="764f3042cd338400ae30589cb0fae8ecaa33ecfc1fea88a93bfe74a59ce411e8" exitCode=0
Sep 16 18:50:20.381273 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:20.381205    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9" event={"ID":"303978d0-6046-43f1-8ba0-5f180332e46e","Type":"ContainerDied","Data":"764f3042cd338400ae30589cb0fae8ecaa33ecfc1fea88a93bfe74a59ce411e8"}
Sep 16 18:50:21.387535 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:21.387495    2578 generic.go:356] "Generic (PLEG): container finished" podID="303978d0-6046-43f1-8ba0-5f180332e46e" containerID="7cfa3660c77aaa31908a9ab588db449681633aac9013f14452c60d44a1159e88" exitCode=0
Sep 16 18:50:21.387535 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:21.387534    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9" event={"ID":"303978d0-6046-43f1-8ba0-5f180332e46e","Type":"ContainerDied","Data":"7cfa3660c77aaa31908a9ab588db449681633aac9013f14452c60d44a1159e88"}
Sep 16 18:50:22.518324 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:22.518300    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9"
Sep 16 18:50:22.555371 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:22.555336    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/303978d0-6046-43f1-8ba0-5f180332e46e-util\") pod \"303978d0-6046-43f1-8ba0-5f180332e46e\" (UID: \"303978d0-6046-43f1-8ba0-5f180332e46e\") "
Sep 16 18:50:22.555561 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:22.555385    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gz9n\" (UniqueName: \"kubernetes.io/projected/303978d0-6046-43f1-8ba0-5f180332e46e-kube-api-access-4gz9n\") pod \"303978d0-6046-43f1-8ba0-5f180332e46e\" (UID: \"303978d0-6046-43f1-8ba0-5f180332e46e\") "
Sep 16 18:50:22.555561 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:22.555441    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/303978d0-6046-43f1-8ba0-5f180332e46e-bundle\") pod \"303978d0-6046-43f1-8ba0-5f180332e46e\" (UID: \"303978d0-6046-43f1-8ba0-5f180332e46e\") "
Sep 16 18:50:22.555980 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:22.555951    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/303978d0-6046-43f1-8ba0-5f180332e46e-bundle" (OuterVolumeSpecName: "bundle") pod "303978d0-6046-43f1-8ba0-5f180332e46e" (UID: "303978d0-6046-43f1-8ba0-5f180332e46e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:50:22.557667 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:22.557635    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/303978d0-6046-43f1-8ba0-5f180332e46e-kube-api-access-4gz9n" (OuterVolumeSpecName: "kube-api-access-4gz9n") pod "303978d0-6046-43f1-8ba0-5f180332e46e" (UID: "303978d0-6046-43f1-8ba0-5f180332e46e"). InnerVolumeSpecName "kube-api-access-4gz9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:50:22.560523 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:22.560499    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/303978d0-6046-43f1-8ba0-5f180332e46e-util" (OuterVolumeSpecName: "util") pod "303978d0-6046-43f1-8ba0-5f180332e46e" (UID: "303978d0-6046-43f1-8ba0-5f180332e46e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:50:22.656297 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:22.656220    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/303978d0-6046-43f1-8ba0-5f180332e46e-util\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:50:22.656297 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:22.656248    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gz9n\" (UniqueName: \"kubernetes.io/projected/303978d0-6046-43f1-8ba0-5f180332e46e-kube-api-access-4gz9n\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:50:22.656297 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:22.656257    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/303978d0-6046-43f1-8ba0-5f180332e46e-bundle\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:50:23.398461 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:23.398429    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9"
Sep 16 18:50:23.398602 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:23.398464    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4kqp9" event={"ID":"303978d0-6046-43f1-8ba0-5f180332e46e","Type":"ContainerDied","Data":"171287d4e2656bcbfb7f00937ca11a5e430dd494b896cfdb2016227b5784c381"}
Sep 16 18:50:23.398602 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:23.398505    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="171287d4e2656bcbfb7f00937ca11a5e430dd494b896cfdb2016227b5784c381"
Sep 16 18:50:28.425045 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:28.425004    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cjlxd" event={"ID":"ded809c0-38fa-4a8f-aae0-ac11509732a0","Type":"ContainerStarted","Data":"60ba092caa538930c6446242bf53a4ac450bba75aa0e7651793cffcc1e57fe3b"}
Sep 16 18:50:28.446973 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:28.446921    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-cjlxd" podStartSLOduration=1.880080769 podStartE2EDuration="9.446904932s" totalImagesPullingTime="7.566824163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:50:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:50:28.443357355 +0000 UTC m=+656.230921512" watchObservedRunningTime="2026-09-16 18:50:28.446904932 +0000 UTC m=+656.234469086"
Sep 16 18:50:29.152380 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.152347    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qsnr"]
Sep 16 18:50:29.152660 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.152648    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="303978d0-6046-43f1-8ba0-5f180332e46e" containerName="pull"
Sep 16 18:50:29.152719 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.152662    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="303978d0-6046-43f1-8ba0-5f180332e46e" containerName="pull"
Sep 16 18:50:29.152719 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.152673    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="303978d0-6046-43f1-8ba0-5f180332e46e" containerName="extract"
Sep 16 18:50:29.152719 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.152678    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="303978d0-6046-43f1-8ba0-5f180332e46e" containerName="extract"
Sep 16 18:50:29.152719 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.152688    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="303978d0-6046-43f1-8ba0-5f180332e46e" containerName="util"
Sep 16 18:50:29.152719 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.152693    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="303978d0-6046-43f1-8ba0-5f180332e46e" containerName="util"
Sep 16 18:50:29.152865 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.152737    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="303978d0-6046-43f1-8ba0-5f180332e46e" containerName="extract"
Sep 16 18:50:29.154458 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.154442    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qsnr"
Sep 16 18:50:29.157274 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.157247    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-n6hpt\""
Sep 16 18:50:29.158010 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.157993    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:50:29.158068 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.157998    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 16 18:50:29.165542 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.165518    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qsnr"]
Sep 16 18:50:29.208748 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.208712    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tppwm\" (UniqueName: \"kubernetes.io/projected/ad015505-63ef-4a46-acee-ff779e3ce4e2-kube-api-access-tppwm\") pod \"openshift-lws-operator-fd8ccff4c-2qsnr\" (UID: \"ad015505-63ef-4a46-acee-ff779e3ce4e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qsnr"
Sep 16 18:50:29.208904 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.208757    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad015505-63ef-4a46-acee-ff779e3ce4e2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2qsnr\" (UID: \"ad015505-63ef-4a46-acee-ff779e3ce4e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qsnr"
Sep 16 18:50:29.310083 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.310038    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tppwm\" (UniqueName: \"kubernetes.io/projected/ad015505-63ef-4a46-acee-ff779e3ce4e2-kube-api-access-tppwm\") pod \"openshift-lws-operator-fd8ccff4c-2qsnr\" (UID: \"ad015505-63ef-4a46-acee-ff779e3ce4e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qsnr"
Sep 16 18:50:29.310302 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.310112    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad015505-63ef-4a46-acee-ff779e3ce4e2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2qsnr\" (UID: \"ad015505-63ef-4a46-acee-ff779e3ce4e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qsnr"
Sep 16 18:50:29.310523 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.310500    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad015505-63ef-4a46-acee-ff779e3ce4e2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2qsnr\" (UID: \"ad015505-63ef-4a46-acee-ff779e3ce4e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qsnr"
Sep 16 18:50:29.325061 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.325033    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tppwm\" (UniqueName: \"kubernetes.io/projected/ad015505-63ef-4a46-acee-ff779e3ce4e2-kube-api-access-tppwm\") pod \"openshift-lws-operator-fd8ccff4c-2qsnr\" (UID: \"ad015505-63ef-4a46-acee-ff779e3ce4e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qsnr"
Sep 16 18:50:29.463465 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.463383    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qsnr"
Sep 16 18:50:29.741944 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:29.741912    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qsnr"]
Sep 16 18:50:29.743477 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:50:29.743450    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad015505_63ef_4a46_acee_ff779e3ce4e2.slice/crio-ae084e4e961751c4d2cdd7daac4f6dfb3026e58450f0c43a29d3735ac01a8831 WatchSource:0}: Error finding container ae084e4e961751c4d2cdd7daac4f6dfb3026e58450f0c43a29d3735ac01a8831: Status 404 returned error can't find the container with id ae084e4e961751c4d2cdd7daac4f6dfb3026e58450f0c43a29d3735ac01a8831
Sep 16 18:50:30.434783 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:30.434744    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qsnr" event={"ID":"ad015505-63ef-4a46-acee-ff779e3ce4e2","Type":"ContainerStarted","Data":"ae084e4e961751c4d2cdd7daac4f6dfb3026e58450f0c43a29d3735ac01a8831"}
Sep 16 18:50:32.450437 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:32.450397    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qsnr" event={"ID":"ad015505-63ef-4a46-acee-ff779e3ce4e2","Type":"ContainerStarted","Data":"ae2edf8ba2e2627c4a872552e52826988288b1f1f6100efedd78aab448cfc7c9"}
Sep 16 18:50:36.444846 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:36.444786    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2qsnr" podStartSLOduration=5.678975243 podStartE2EDuration="7.444768934s" totalImagesPullingTime="1.765793691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:50:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:50:32.495205901 +0000 UTC m=+660.282770053" watchObservedRunningTime="2026-09-16 18:50:36.444768934 +0000 UTC m=+664.232333088"
Sep 16 18:50:36.445709 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:36.445690    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n8v84"]
Sep 16 18:50:36.449406 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:36.449390    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-n8v84"
Sep 16 18:50:36.454112 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:36.454094    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-mtx87\""
Sep 16 18:50:36.467143 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:36.467114    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n8v84"]
Sep 16 18:50:36.569457 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:36.569413    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9259d12-2263-4ad2-a839-d68fa5629e20-bound-sa-token\") pod \"cert-manager-55b4d954d5-n8v84\" (UID: \"e9259d12-2263-4ad2-a839-d68fa5629e20\") " pod="cert-manager/cert-manager-55b4d954d5-n8v84"
Sep 16 18:50:36.569641 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:36.569528    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x22z8\" (UniqueName: \"kubernetes.io/projected/e9259d12-2263-4ad2-a839-d68fa5629e20-kube-api-access-x22z8\") pod \"cert-manager-55b4d954d5-n8v84\" (UID: \"e9259d12-2263-4ad2-a839-d68fa5629e20\") " pod="cert-manager/cert-manager-55b4d954d5-n8v84"
Sep 16 18:50:36.670132 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:36.670095    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9259d12-2263-4ad2-a839-d68fa5629e20-bound-sa-token\") pod \"cert-manager-55b4d954d5-n8v84\" (UID: \"e9259d12-2263-4ad2-a839-d68fa5629e20\") " pod="cert-manager/cert-manager-55b4d954d5-n8v84"
Sep 16 18:50:36.670337 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:36.670154    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x22z8\" (UniqueName: \"kubernetes.io/projected/e9259d12-2263-4ad2-a839-d68fa5629e20-kube-api-access-x22z8\") pod \"cert-manager-55b4d954d5-n8v84\" (UID: \"e9259d12-2263-4ad2-a839-d68fa5629e20\") " pod="cert-manager/cert-manager-55b4d954d5-n8v84"
Sep 16 18:50:36.680500 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:36.680472    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9259d12-2263-4ad2-a839-d68fa5629e20-bound-sa-token\") pod \"cert-manager-55b4d954d5-n8v84\" (UID: \"e9259d12-2263-4ad2-a839-d68fa5629e20\") " pod="cert-manager/cert-manager-55b4d954d5-n8v84"
Sep 16 18:50:36.680500 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:36.680492    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x22z8\" (UniqueName: \"kubernetes.io/projected/e9259d12-2263-4ad2-a839-d68fa5629e20-kube-api-access-x22z8\") pod \"cert-manager-55b4d954d5-n8v84\" (UID: \"e9259d12-2263-4ad2-a839-d68fa5629e20\") " pod="cert-manager/cert-manager-55b4d954d5-n8v84"
Sep 16 18:50:36.757763 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:36.757679    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-n8v84"
Sep 16 18:50:37.035987 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:37.035787    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-n8v84"]
Sep 16 18:50:37.038784 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:50:37.038752    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9259d12_2263_4ad2_a839_d68fa5629e20.slice/crio-9dcfebcf3e9944303ae6e8e2ce75c910a96fc34bd5948de99b2eb1b708e1c2bf WatchSource:0}: Error finding container 9dcfebcf3e9944303ae6e8e2ce75c910a96fc34bd5948de99b2eb1b708e1c2bf: Status 404 returned error can't find the container with id 9dcfebcf3e9944303ae6e8e2ce75c910a96fc34bd5948de99b2eb1b708e1c2bf
Sep 16 18:50:37.477375 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:37.477337    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-n8v84" event={"ID":"e9259d12-2263-4ad2-a839-d68fa5629e20","Type":"ContainerStarted","Data":"9b13fec7a9b4f50786c812ac937349bb12e73feb93198629fd0bab941760cf19"}
Sep 16 18:50:37.477767 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:37.477386    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-n8v84" event={"ID":"e9259d12-2263-4ad2-a839-d68fa5629e20","Type":"ContainerStarted","Data":"9dcfebcf3e9944303ae6e8e2ce75c910a96fc34bd5948de99b2eb1b708e1c2bf"}
Sep 16 18:50:37.500244 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:37.500196    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-n8v84" podStartSLOduration=1.500165542 podStartE2EDuration="1.500165542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:50:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:50:37.49784153 +0000 UTC m=+665.285405685" watchObservedRunningTime="2026-09-16 18:50:37.500165542 +0000 UTC m=+665.287729697"
Sep 16 18:50:38.007635 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.007599    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2"]
Sep 16 18:50:38.010250 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.010226    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2"
Sep 16 18:50:38.015635 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.015615    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 18:50:38.015770 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.015721    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 18:50:38.015838 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.015804    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5tnch\""
Sep 16 18:50:38.023069 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.023046    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2"]
Sep 16 18:50:38.080690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.080650    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0357a419-a3d6-4e72-96c7-194114c877c1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2\" (UID: \"0357a419-a3d6-4e72-96c7-194114c877c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2"
Sep 16 18:50:38.080690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.080694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5865\" (UniqueName: \"kubernetes.io/projected/0357a419-a3d6-4e72-96c7-194114c877c1-kube-api-access-z5865\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2\" (UID: \"0357a419-a3d6-4e72-96c7-194114c877c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2"
Sep 16 18:50:38.080898 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.080816    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0357a419-a3d6-4e72-96c7-194114c877c1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2\" (UID: \"0357a419-a3d6-4e72-96c7-194114c877c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2"
Sep 16 18:50:38.181646 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.181611    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0357a419-a3d6-4e72-96c7-194114c877c1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2\" (UID: \"0357a419-a3d6-4e72-96c7-194114c877c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2"
Sep 16 18:50:38.181832 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.181660    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0357a419-a3d6-4e72-96c7-194114c877c1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2\" (UID: \"0357a419-a3d6-4e72-96c7-194114c877c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2"
Sep 16 18:50:38.181832 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.181684    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5865\" (UniqueName: \"kubernetes.io/projected/0357a419-a3d6-4e72-96c7-194114c877c1-kube-api-access-z5865\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2\" (UID: \"0357a419-a3d6-4e72-96c7-194114c877c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2"
Sep 16 18:50:38.182018 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.181995    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0357a419-a3d6-4e72-96c7-194114c877c1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2\" (UID: \"0357a419-a3d6-4e72-96c7-194114c877c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2"
Sep 16 18:50:38.182084 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.182042    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0357a419-a3d6-4e72-96c7-194114c877c1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2\" (UID: \"0357a419-a3d6-4e72-96c7-194114c877c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2"
Sep 16 18:50:38.191299 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.191279    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5865\" (UniqueName: \"kubernetes.io/projected/0357a419-a3d6-4e72-96c7-194114c877c1-kube-api-access-z5865\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2\" (UID: \"0357a419-a3d6-4e72-96c7-194114c877c1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2"
Sep 16 18:50:38.319113 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.319030    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2"
Sep 16 18:50:38.587945 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:38.587916    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2"]
Sep 16 18:50:38.590379 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:50:38.590354    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0357a419_a3d6_4e72_96c7_194114c877c1.slice/crio-bd8ed73054359d7f2b25b6225d981cb348e84bf23cb3f3579622d44f440facaa WatchSource:0}: Error finding container bd8ed73054359d7f2b25b6225d981cb348e84bf23cb3f3579622d44f440facaa: Status 404 returned error can't find the container with id bd8ed73054359d7f2b25b6225d981cb348e84bf23cb3f3579622d44f440facaa
Sep 16 18:50:39.487259 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:39.487224    2578 generic.go:356] "Generic (PLEG): container finished" podID="0357a419-a3d6-4e72-96c7-194114c877c1" containerID="fed87569450ff08753a36a0152620b5bdce47d79eb325993fbed893057ae0acb" exitCode=0
Sep 16 18:50:39.487427 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:39.487313    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2" event={"ID":"0357a419-a3d6-4e72-96c7-194114c877c1","Type":"ContainerDied","Data":"fed87569450ff08753a36a0152620b5bdce47d79eb325993fbed893057ae0acb"}
Sep 16 18:50:39.487427 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:39.487390    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2" event={"ID":"0357a419-a3d6-4e72-96c7-194114c877c1","Type":"ContainerStarted","Data":"bd8ed73054359d7f2b25b6225d981cb348e84bf23cb3f3579622d44f440facaa"}
Sep 16 18:50:40.494069 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:40.494035    2578 generic.go:356] "Generic (PLEG): container finished" podID="0357a419-a3d6-4e72-96c7-194114c877c1" containerID="e2dcc2178947c73d3c26987050ad9390691cd7b54cbe28f62f5a68f821c8fcf9" exitCode=0
Sep 16 18:50:40.494467 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:40.494114    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2" event={"ID":"0357a419-a3d6-4e72-96c7-194114c877c1","Type":"ContainerDied","Data":"e2dcc2178947c73d3c26987050ad9390691cd7b54cbe28f62f5a68f821c8fcf9"}
Sep 16 18:50:41.501088 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:41.501056    2578 generic.go:356] "Generic (PLEG): container finished" podID="0357a419-a3d6-4e72-96c7-194114c877c1" containerID="5ecc490029a91f59942e45397dab42919bfbf92f87c189d548b6296ceaf97803" exitCode=0
Sep 16 18:50:41.501501 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:41.501140    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2" event={"ID":"0357a419-a3d6-4e72-96c7-194114c877c1","Type":"ContainerDied","Data":"5ecc490029a91f59942e45397dab42919bfbf92f87c189d548b6296ceaf97803"}
Sep 16 18:50:42.630601 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:42.630578    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2"
Sep 16 18:50:42.720969 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:42.720934    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5865\" (UniqueName: \"kubernetes.io/projected/0357a419-a3d6-4e72-96c7-194114c877c1-kube-api-access-z5865\") pod \"0357a419-a3d6-4e72-96c7-194114c877c1\" (UID: \"0357a419-a3d6-4e72-96c7-194114c877c1\") "
Sep 16 18:50:42.721129 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:42.721003    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0357a419-a3d6-4e72-96c7-194114c877c1-bundle\") pod \"0357a419-a3d6-4e72-96c7-194114c877c1\" (UID: \"0357a419-a3d6-4e72-96c7-194114c877c1\") "
Sep 16 18:50:42.721129 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:42.721050    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0357a419-a3d6-4e72-96c7-194114c877c1-util\") pod \"0357a419-a3d6-4e72-96c7-194114c877c1\" (UID: \"0357a419-a3d6-4e72-96c7-194114c877c1\") "
Sep 16 18:50:42.721916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:42.721883    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0357a419-a3d6-4e72-96c7-194114c877c1-bundle" (OuterVolumeSpecName: "bundle") pod "0357a419-a3d6-4e72-96c7-194114c877c1" (UID: "0357a419-a3d6-4e72-96c7-194114c877c1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:50:42.723082 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:42.723054    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0357a419-a3d6-4e72-96c7-194114c877c1-kube-api-access-z5865" (OuterVolumeSpecName: "kube-api-access-z5865") pod "0357a419-a3d6-4e72-96c7-194114c877c1" (UID: "0357a419-a3d6-4e72-96c7-194114c877c1"). InnerVolumeSpecName "kube-api-access-z5865". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:50:42.726821 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:42.726789    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0357a419-a3d6-4e72-96c7-194114c877c1-util" (OuterVolumeSpecName: "util") pod "0357a419-a3d6-4e72-96c7-194114c877c1" (UID: "0357a419-a3d6-4e72-96c7-194114c877c1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:50:42.821798 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:42.821710    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5865\" (UniqueName: \"kubernetes.io/projected/0357a419-a3d6-4e72-96c7-194114c877c1-kube-api-access-z5865\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:50:42.821798 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:42.821736    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0357a419-a3d6-4e72-96c7-194114c877c1-bundle\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:50:42.821798 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:42.821753    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0357a419-a3d6-4e72-96c7-194114c877c1-util\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:50:43.512144 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:43.512105    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2" event={"ID":"0357a419-a3d6-4e72-96c7-194114c877c1","Type":"ContainerDied","Data":"bd8ed73054359d7f2b25b6225d981cb348e84bf23cb3f3579622d44f440facaa"}
Sep 16 18:50:43.512333 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:43.512151    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afshzq2"
Sep 16 18:50:43.512333 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:43.512159    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd8ed73054359d7f2b25b6225d981cb348e84bf23cb3f3579622d44f440facaa"
Sep 16 18:50:52.626050 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.626015    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l"]
Sep 16 18:50:52.626442 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.626377    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0357a419-a3d6-4e72-96c7-194114c877c1" containerName="extract"
Sep 16 18:50:52.626442 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.626391    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0357a419-a3d6-4e72-96c7-194114c877c1" containerName="extract"
Sep 16 18:50:52.626442 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.626406    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0357a419-a3d6-4e72-96c7-194114c877c1" containerName="pull"
Sep 16 18:50:52.626442 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.626412    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0357a419-a3d6-4e72-96c7-194114c877c1" containerName="pull"
Sep 16 18:50:52.626442 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.626421    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0357a419-a3d6-4e72-96c7-194114c877c1" containerName="util"
Sep 16 18:50:52.626442 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.626427    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0357a419-a3d6-4e72-96c7-194114c877c1" containerName="util"
Sep 16 18:50:52.626623 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.626485    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0357a419-a3d6-4e72-96c7-194114c877c1" containerName="extract"
Sep 16 18:50:52.633984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.633965    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l"
Sep 16 18:50:52.636440 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.636403    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5tnch\""
Sep 16 18:50:52.636568 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.636459    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 18:50:52.637047 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.637027    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 18:50:52.652914 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.652887    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l"]
Sep 16 18:50:52.712111 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.712063    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b2088f7-429c-4964-bb8e-a9e5deac0856-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l\" (UID: \"5b2088f7-429c-4964-bb8e-a9e5deac0856\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l"
Sep 16 18:50:52.712111 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.712107    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b2088f7-429c-4964-bb8e-a9e5deac0856-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l\" (UID: \"5b2088f7-429c-4964-bb8e-a9e5deac0856\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l"
Sep 16 18:50:52.712372 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.712274    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4g2s\" (UniqueName: \"kubernetes.io/projected/5b2088f7-429c-4964-bb8e-a9e5deac0856-kube-api-access-b4g2s\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l\" (UID: \"5b2088f7-429c-4964-bb8e-a9e5deac0856\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l"
Sep 16 18:50:52.812709 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.812676    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4g2s\" (UniqueName: \"kubernetes.io/projected/5b2088f7-429c-4964-bb8e-a9e5deac0856-kube-api-access-b4g2s\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l\" (UID: \"5b2088f7-429c-4964-bb8e-a9e5deac0856\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l"
Sep 16 18:50:52.812884 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.812725    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b2088f7-429c-4964-bb8e-a9e5deac0856-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l\" (UID: \"5b2088f7-429c-4964-bb8e-a9e5deac0856\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l"
Sep 16 18:50:52.812884 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.812753    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b2088f7-429c-4964-bb8e-a9e5deac0856-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l\" (UID: \"5b2088f7-429c-4964-bb8e-a9e5deac0856\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l"
Sep 16 18:50:52.813208 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.813158    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b2088f7-429c-4964-bb8e-a9e5deac0856-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l\" (UID: \"5b2088f7-429c-4964-bb8e-a9e5deac0856\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l"
Sep 16 18:50:52.813328 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.813223    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b2088f7-429c-4964-bb8e-a9e5deac0856-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l\" (UID: \"5b2088f7-429c-4964-bb8e-a9e5deac0856\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l"
Sep 16 18:50:52.825019 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.824986    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4g2s\" (UniqueName: \"kubernetes.io/projected/5b2088f7-429c-4964-bb8e-a9e5deac0856-kube-api-access-b4g2s\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l\" (UID: \"5b2088f7-429c-4964-bb8e-a9e5deac0856\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l"
Sep 16 18:50:52.942993 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:52.942895    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l"
Sep 16 18:50:53.218329 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:53.218301    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l"]
Sep 16 18:50:53.220471 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:50:53.220440    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b2088f7_429c_4964_bb8e_a9e5deac0856.slice/crio-0e98bdb637bd6dfa48dba1da9028ecb4ee1ad13601395b80cff11d30d3801e7c WatchSource:0}: Error finding container 0e98bdb637bd6dfa48dba1da9028ecb4ee1ad13601395b80cff11d30d3801e7c: Status 404 returned error can't find the container with id 0e98bdb637bd6dfa48dba1da9028ecb4ee1ad13601395b80cff11d30d3801e7c
Sep 16 18:50:53.560024 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:53.559938    2578 generic.go:356] "Generic (PLEG): container finished" podID="5b2088f7-429c-4964-bb8e-a9e5deac0856" containerID="fde687eee404a2ecd30306299bfaa1beaa5d3cb9db197ea3be6ae2e3da13fd22" exitCode=0
Sep 16 18:50:53.560164 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:53.560018    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l" event={"ID":"5b2088f7-429c-4964-bb8e-a9e5deac0856","Type":"ContainerDied","Data":"fde687eee404a2ecd30306299bfaa1beaa5d3cb9db197ea3be6ae2e3da13fd22"}
Sep 16 18:50:53.560164 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:53.560069    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l" event={"ID":"5b2088f7-429c-4964-bb8e-a9e5deac0856","Type":"ContainerStarted","Data":"0e98bdb637bd6dfa48dba1da9028ecb4ee1ad13601395b80cff11d30d3801e7c"}
Sep 16 18:50:54.210066 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.210036    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"]
Sep 16 18:50:54.213392 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.213376    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.217098 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.217075    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 16 18:50:54.217258 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.217137    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 16 18:50:54.217258 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.217212    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 16 18:50:54.217502 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.217486    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 16 18:50:54.217604 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.217589    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gbmdk\""
Sep 16 18:50:54.233193 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.233139    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"]
Sep 16 18:50:54.328134 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.328093    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0a308b28-fbda-4df4-b85c-ec0cf9b956bc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85745479c7-5jz79\" (UID: \"0a308b28-fbda-4df4-b85c-ec0cf9b956bc\") " pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.328351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.328140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a308b28-fbda-4df4-b85c-ec0cf9b956bc-tmp\") pod \"opendatahub-operator-controller-manager-85745479c7-5jz79\" (UID: \"0a308b28-fbda-4df4-b85c-ec0cf9b956bc\") " pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.328351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.328233    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6szsz\" (UniqueName: \"kubernetes.io/projected/0a308b28-fbda-4df4-b85c-ec0cf9b956bc-kube-api-access-6szsz\") pod \"opendatahub-operator-controller-manager-85745479c7-5jz79\" (UID: \"0a308b28-fbda-4df4-b85c-ec0cf9b956bc\") " pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.328351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.328289    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0a308b28-fbda-4df4-b85c-ec0cf9b956bc-manifests\") pod \"opendatahub-operator-controller-manager-85745479c7-5jz79\" (UID: \"0a308b28-fbda-4df4-b85c-ec0cf9b956bc\") " pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.328351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.328348    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a308b28-fbda-4df4-b85c-ec0cf9b956bc-webhook-cert\") pod \"opendatahub-operator-controller-manager-85745479c7-5jz79\" (UID: \"0a308b28-fbda-4df4-b85c-ec0cf9b956bc\") " pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.428905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.428867    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0a308b28-fbda-4df4-b85c-ec0cf9b956bc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85745479c7-5jz79\" (UID: \"0a308b28-fbda-4df4-b85c-ec0cf9b956bc\") " pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.429078 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.428922    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a308b28-fbda-4df4-b85c-ec0cf9b956bc-tmp\") pod \"opendatahub-operator-controller-manager-85745479c7-5jz79\" (UID: \"0a308b28-fbda-4df4-b85c-ec0cf9b956bc\") " pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.429078 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.428958    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6szsz\" (UniqueName: \"kubernetes.io/projected/0a308b28-fbda-4df4-b85c-ec0cf9b956bc-kube-api-access-6szsz\") pod \"opendatahub-operator-controller-manager-85745479c7-5jz79\" (UID: \"0a308b28-fbda-4df4-b85c-ec0cf9b956bc\") " pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.429078 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.429006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0a308b28-fbda-4df4-b85c-ec0cf9b956bc-manifests\") pod \"opendatahub-operator-controller-manager-85745479c7-5jz79\" (UID: \"0a308b28-fbda-4df4-b85c-ec0cf9b956bc\") " pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.429078 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.429062    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a308b28-fbda-4df4-b85c-ec0cf9b956bc-webhook-cert\") pod \"opendatahub-operator-controller-manager-85745479c7-5jz79\" (UID: \"0a308b28-fbda-4df4-b85c-ec0cf9b956bc\") " pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.429522 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.429478    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0a308b28-fbda-4df4-b85c-ec0cf9b956bc-manifests\") pod \"opendatahub-operator-controller-manager-85745479c7-5jz79\" (UID: \"0a308b28-fbda-4df4-b85c-ec0cf9b956bc\") " pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.431364 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.431335    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a308b28-fbda-4df4-b85c-ec0cf9b956bc-tmp\") pod \"opendatahub-operator-controller-manager-85745479c7-5jz79\" (UID: \"0a308b28-fbda-4df4-b85c-ec0cf9b956bc\") " pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.431560 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.431544    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0a308b28-fbda-4df4-b85c-ec0cf9b956bc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85745479c7-5jz79\" (UID: \"0a308b28-fbda-4df4-b85c-ec0cf9b956bc\") " pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.431607 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.431563    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a308b28-fbda-4df4-b85c-ec0cf9b956bc-webhook-cert\") pod \"opendatahub-operator-controller-manager-85745479c7-5jz79\" (UID: \"0a308b28-fbda-4df4-b85c-ec0cf9b956bc\") " pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.443258 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.443230    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6szsz\" (UniqueName: \"kubernetes.io/projected/0a308b28-fbda-4df4-b85c-ec0cf9b956bc-kube-api-access-6szsz\") pod \"opendatahub-operator-controller-manager-85745479c7-5jz79\" (UID: \"0a308b28-fbda-4df4-b85c-ec0cf9b956bc\") " pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.527323 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.527280    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:50:54.567740 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.567709    2578 generic.go:356] "Generic (PLEG): container finished" podID="5b2088f7-429c-4964-bb8e-a9e5deac0856" containerID="1799802e3ab7c534f60380235bda82ecfc71b49c069cab81fddeddc05188fb4e" exitCode=0
Sep 16 18:50:54.567922 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.567751    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l" event={"ID":"5b2088f7-429c-4964-bb8e-a9e5deac0856","Type":"ContainerDied","Data":"1799802e3ab7c534f60380235bda82ecfc71b49c069cab81fddeddc05188fb4e"}
Sep 16 18:50:54.834532 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:54.834497    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"]
Sep 16 18:50:54.836434 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:50:54.836402    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a308b28_fbda_4df4_b85c_ec0cf9b956bc.slice/crio-fba9da4633cde94800612f0c20d8abea16b963bdedb2eab38b651d1a397e822e WatchSource:0}: Error finding container fba9da4633cde94800612f0c20d8abea16b963bdedb2eab38b651d1a397e822e: Status 404 returned error can't find the container with id fba9da4633cde94800612f0c20d8abea16b963bdedb2eab38b651d1a397e822e
Sep 16 18:50:55.577805 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:55.577775    2578 generic.go:356] "Generic (PLEG): container finished" podID="5b2088f7-429c-4964-bb8e-a9e5deac0856" containerID="d78022e9405033988113a0256e666ac5d8601234434873bc8f985764659767a5" exitCode=0
Sep 16 18:50:55.578272 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:55.577850    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l" event={"ID":"5b2088f7-429c-4964-bb8e-a9e5deac0856","Type":"ContainerDied","Data":"d78022e9405033988113a0256e666ac5d8601234434873bc8f985764659767a5"}
Sep 16 18:50:55.579399 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:55.579372    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79" event={"ID":"0a308b28-fbda-4df4-b85c-ec0cf9b956bc","Type":"ContainerStarted","Data":"fba9da4633cde94800612f0c20d8abea16b963bdedb2eab38b651d1a397e822e"}
Sep 16 18:50:57.091987 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:57.091953    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l"
Sep 16 18:50:57.254887 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:57.254843    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4g2s\" (UniqueName: \"kubernetes.io/projected/5b2088f7-429c-4964-bb8e-a9e5deac0856-kube-api-access-b4g2s\") pod \"5b2088f7-429c-4964-bb8e-a9e5deac0856\" (UID: \"5b2088f7-429c-4964-bb8e-a9e5deac0856\") "
Sep 16 18:50:57.255054 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:57.254932    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b2088f7-429c-4964-bb8e-a9e5deac0856-bundle\") pod \"5b2088f7-429c-4964-bb8e-a9e5deac0856\" (UID: \"5b2088f7-429c-4964-bb8e-a9e5deac0856\") "
Sep 16 18:50:57.255054 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:57.255005    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b2088f7-429c-4964-bb8e-a9e5deac0856-util\") pod \"5b2088f7-429c-4964-bb8e-a9e5deac0856\" (UID: \"5b2088f7-429c-4964-bb8e-a9e5deac0856\") "
Sep 16 18:50:57.255856 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:57.255785    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b2088f7-429c-4964-bb8e-a9e5deac0856-bundle" (OuterVolumeSpecName: "bundle") pod "5b2088f7-429c-4964-bb8e-a9e5deac0856" (UID: "5b2088f7-429c-4964-bb8e-a9e5deac0856"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:50:57.257706 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:57.257670    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b2088f7-429c-4964-bb8e-a9e5deac0856-kube-api-access-b4g2s" (OuterVolumeSpecName: "kube-api-access-b4g2s") pod "5b2088f7-429c-4964-bb8e-a9e5deac0856" (UID: "5b2088f7-429c-4964-bb8e-a9e5deac0856"). InnerVolumeSpecName "kube-api-access-b4g2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:50:57.262944 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:57.262906    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b2088f7-429c-4964-bb8e-a9e5deac0856-util" (OuterVolumeSpecName: "util") pod "5b2088f7-429c-4964-bb8e-a9e5deac0856" (UID: "5b2088f7-429c-4964-bb8e-a9e5deac0856"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:50:57.356404 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:57.356364    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b2088f7-429c-4964-bb8e-a9e5deac0856-util\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:50:57.356404 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:57.356406    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4g2s\" (UniqueName: \"kubernetes.io/projected/5b2088f7-429c-4964-bb8e-a9e5deac0856-kube-api-access-b4g2s\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:50:57.356638 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:57.356423    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b2088f7-429c-4964-bb8e-a9e5deac0856-bundle\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:50:57.596681 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:57.596650    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l"
Sep 16 18:50:57.596681 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:57.596663    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0bgt8l" event={"ID":"5b2088f7-429c-4964-bb8e-a9e5deac0856","Type":"ContainerDied","Data":"0e98bdb637bd6dfa48dba1da9028ecb4ee1ad13601395b80cff11d30d3801e7c"}
Sep 16 18:50:57.596898 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:57.596711    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e98bdb637bd6dfa48dba1da9028ecb4ee1ad13601395b80cff11d30d3801e7c"
Sep 16 18:50:58.606114 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:58.606076    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79" event={"ID":"0a308b28-fbda-4df4-b85c-ec0cf9b956bc","Type":"ContainerStarted","Data":"f2cb1c13598cfb2ef357c704f8bc4cbe99eb174beb4a1bf636467a9feba1652c"}
Sep 16 18:50:59.612223 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:59.612193    2578 generic.go:356] "Generic (PLEG): container finished" podID="0a308b28-fbda-4df4-b85c-ec0cf9b956bc" containerID="f2cb1c13598cfb2ef357c704f8bc4cbe99eb174beb4a1bf636467a9feba1652c" exitCode=0
Sep 16 18:50:59.612643 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:50:59.612280    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79" event={"ID":"0a308b28-fbda-4df4-b85c-ec0cf9b956bc","Type":"ContainerDied","Data":"f2cb1c13598cfb2ef357c704f8bc4cbe99eb174beb4a1bf636467a9feba1652c"}
Sep 16 18:51:00.618836 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:00.618804    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79" event={"ID":"0a308b28-fbda-4df4-b85c-ec0cf9b956bc","Type":"ContainerStarted","Data":"188325a302cf0780e5cb02145d639fdbec3d9575ad05bd50727b98af4f6837b0"}
Sep 16 18:51:00.619219 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:00.618933    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:51:00.641404 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:00.641349    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79" podStartSLOduration=2.012457027 podStartE2EDuration="6.641331036s" totalImagesPullingTime="3.580189783s" totalInitContainerRuntime="1.048684226s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:50:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:51:00.639709526 +0000 UTC m=+688.427273711" watchObservedRunningTime="2026-09-16 18:51:00.641331036 +0000 UTC m=+688.428895191"
Sep 16 18:51:11.626033 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:11.625998    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-85745479c7-5jz79"
Sep 16 18:51:24.270049 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.270006    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf"]
Sep 16 18:51:24.270545 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.270524    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b2088f7-429c-4964-bb8e-a9e5deac0856" containerName="extract"
Sep 16 18:51:24.270618 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.270548    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2088f7-429c-4964-bb8e-a9e5deac0856" containerName="extract"
Sep 16 18:51:24.270618 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.270566    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b2088f7-429c-4964-bb8e-a9e5deac0856" containerName="pull"
Sep 16 18:51:24.270618 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.270574    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2088f7-429c-4964-bb8e-a9e5deac0856" containerName="pull"
Sep 16 18:51:24.270618 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.270600    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b2088f7-429c-4964-bb8e-a9e5deac0856" containerName="util"
Sep 16 18:51:24.270618 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.270608    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2088f7-429c-4964-bb8e-a9e5deac0856" containerName="util"
Sep 16 18:51:24.270851 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.270699    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b2088f7-429c-4964-bb8e-a9e5deac0856" containerName="extract"
Sep 16 18:51:24.278707 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.278683    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf"
Sep 16 18:51:24.284551 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.284522    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 16 18:51:24.285892 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.285475    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 16 18:51:24.285892 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.285753    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-t6q88\""
Sep 16 18:51:24.286420 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.286398    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 18:51:24.286713 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.286431    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 18:51:24.289601 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.289578    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf"]
Sep 16 18:51:24.385949 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.385920    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqr5g\" (UniqueName: \"kubernetes.io/projected/48194838-f8d0-4b1a-a3b3-0ad28d1076a1-kube-api-access-sqr5g\") pod \"kube-auth-proxy-7d78bff8dd-bsvjf\" (UID: \"48194838-f8d0-4b1a-a3b3-0ad28d1076a1\") " pod="openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf"
Sep 16 18:51:24.385949 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.385955    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48194838-f8d0-4b1a-a3b3-0ad28d1076a1-tls-certs\") pod \"kube-auth-proxy-7d78bff8dd-bsvjf\" (UID: \"48194838-f8d0-4b1a-a3b3-0ad28d1076a1\") " pod="openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf"
Sep 16 18:51:24.386201 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.385994    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48194838-f8d0-4b1a-a3b3-0ad28d1076a1-tmp\") pod \"kube-auth-proxy-7d78bff8dd-bsvjf\" (UID: \"48194838-f8d0-4b1a-a3b3-0ad28d1076a1\") " pod="openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf"
Sep 16 18:51:24.486693 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.486654    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48194838-f8d0-4b1a-a3b3-0ad28d1076a1-tmp\") pod \"kube-auth-proxy-7d78bff8dd-bsvjf\" (UID: \"48194838-f8d0-4b1a-a3b3-0ad28d1076a1\") " pod="openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf"
Sep 16 18:51:24.486878 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.486725    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqr5g\" (UniqueName: \"kubernetes.io/projected/48194838-f8d0-4b1a-a3b3-0ad28d1076a1-kube-api-access-sqr5g\") pod \"kube-auth-proxy-7d78bff8dd-bsvjf\" (UID: \"48194838-f8d0-4b1a-a3b3-0ad28d1076a1\") " pod="openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf"
Sep 16 18:51:24.486878 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.486752    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48194838-f8d0-4b1a-a3b3-0ad28d1076a1-tls-certs\") pod \"kube-auth-proxy-7d78bff8dd-bsvjf\" (UID: \"48194838-f8d0-4b1a-a3b3-0ad28d1076a1\") " pod="openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf"
Sep 16 18:51:24.486878 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:51:24.486868    2578 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 16 18:51:24.487041 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:51:24.486937    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48194838-f8d0-4b1a-a3b3-0ad28d1076a1-tls-certs podName:48194838-f8d0-4b1a-a3b3-0ad28d1076a1 nodeName:}" failed. No retries permitted until 2026-09-16 18:51:24.986919198 +0000 UTC m=+712.774483349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/48194838-f8d0-4b1a-a3b3-0ad28d1076a1-tls-certs") pod "kube-auth-proxy-7d78bff8dd-bsvjf" (UID: "48194838-f8d0-4b1a-a3b3-0ad28d1076a1") : secret "kube-auth-proxy-tls" not found
Sep 16 18:51:24.489010 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.488987    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48194838-f8d0-4b1a-a3b3-0ad28d1076a1-tmp\") pod \"kube-auth-proxy-7d78bff8dd-bsvjf\" (UID: \"48194838-f8d0-4b1a-a3b3-0ad28d1076a1\") " pod="openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf"
Sep 16 18:51:24.505565 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.505542    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqr5g\" (UniqueName: \"kubernetes.io/projected/48194838-f8d0-4b1a-a3b3-0ad28d1076a1-kube-api-access-sqr5g\") pod \"kube-auth-proxy-7d78bff8dd-bsvjf\" (UID: \"48194838-f8d0-4b1a-a3b3-0ad28d1076a1\") " pod="openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf"
Sep 16 18:51:24.990949 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.990865    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48194838-f8d0-4b1a-a3b3-0ad28d1076a1-tls-certs\") pod \"kube-auth-proxy-7d78bff8dd-bsvjf\" (UID: \"48194838-f8d0-4b1a-a3b3-0ad28d1076a1\") " pod="openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf"
Sep 16 18:51:24.993433 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:24.993410    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48194838-f8d0-4b1a-a3b3-0ad28d1076a1-tls-certs\") pod \"kube-auth-proxy-7d78bff8dd-bsvjf\" (UID: \"48194838-f8d0-4b1a-a3b3-0ad28d1076a1\") " pod="openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf"
Sep 16 18:51:25.190035 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:25.190005    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf"
Sep 16 18:51:25.490468 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:25.490443    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf"]
Sep 16 18:51:25.493331 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:51:25.493250    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48194838_f8d0_4b1a_a3b3_0ad28d1076a1.slice/crio-450f5f41b4b6385d43c5db823e37101ce7343bc53681257f2935fcddb49e6415 WatchSource:0}: Error finding container 450f5f41b4b6385d43c5db823e37101ce7343bc53681257f2935fcddb49e6415: Status 404 returned error can't find the container with id 450f5f41b4b6385d43c5db823e37101ce7343bc53681257f2935fcddb49e6415
Sep 16 18:51:25.740423 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:25.740385    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf" event={"ID":"48194838-f8d0-4b1a-a3b3-0ad28d1076a1","Type":"ContainerStarted","Data":"450f5f41b4b6385d43c5db823e37101ce7343bc53681257f2935fcddb49e6415"}
Sep 16 18:51:28.758098 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:28.758060    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf" event={"ID":"48194838-f8d0-4b1a-a3b3-0ad28d1076a1","Type":"ContainerStarted","Data":"cb35e8ad119c2e021e5e3f5387bfd6b9c3af3dfc650131f3dd869d712c793c49"}
Sep 16 18:51:28.777889 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:28.777760    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7d78bff8dd-bsvjf" podStartSLOduration=2.002752653 podStartE2EDuration="4.777744936s" totalImagesPullingTime="2.774992283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:51:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:51:28.775903579 +0000 UTC m=+716.563467756" watchObservedRunningTime="2026-09-16 18:51:28.777744936 +0000 UTC m=+716.565309091"
Sep 16 18:51:31.995623 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:31.995590    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"]
Sep 16 18:51:32.003038 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.003015    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.005521 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.005497    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 16 18:51:32.005810 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.005773    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 16 18:51:32.005869 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.005851    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rzjpm\""
Sep 16 18:51:32.005920 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.005905    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 16 18:51:32.006206 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.006190    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 16 18:51:32.015780 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.015758    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"]
Sep 16 18:51:32.063222 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.063157    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ac21262d-9d0c-42cf-ad35-50819dfdeeed-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.063415 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.063240    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac21262d-9d0c-42cf-ad35-50819dfdeeed-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.063415 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.063267    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ac21262d-9d0c-42cf-ad35-50819dfdeeed-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.063415 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.063361    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxtzt\" (UniqueName: \"kubernetes.io/projected/ac21262d-9d0c-42cf-ad35-50819dfdeeed-kube-api-access-nxtzt\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.063549 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.063425    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ac21262d-9d0c-42cf-ad35-50819dfdeeed-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.063549 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.063456    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac21262d-9d0c-42cf-ad35-50819dfdeeed-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.063549 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.063476    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ac21262d-9d0c-42cf-ad35-50819dfdeeed-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.164514 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.164477    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxtzt\" (UniqueName: \"kubernetes.io/projected/ac21262d-9d0c-42cf-ad35-50819dfdeeed-kube-api-access-nxtzt\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.164713 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.164523    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ac21262d-9d0c-42cf-ad35-50819dfdeeed-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.164713 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.164543    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac21262d-9d0c-42cf-ad35-50819dfdeeed-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.164713 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.164561    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ac21262d-9d0c-42cf-ad35-50819dfdeeed-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.164713 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.164590    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ac21262d-9d0c-42cf-ad35-50819dfdeeed-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.164713 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.164617    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac21262d-9d0c-42cf-ad35-50819dfdeeed-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.164713 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.164632    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ac21262d-9d0c-42cf-ad35-50819dfdeeed-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.165254 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.165225    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ac21262d-9d0c-42cf-ad35-50819dfdeeed-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.167066 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.167036    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac21262d-9d0c-42cf-ad35-50819dfdeeed-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.167197 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.167078    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ac21262d-9d0c-42cf-ad35-50819dfdeeed-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.167197 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.167075    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ac21262d-9d0c-42cf-ad35-50819dfdeeed-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.167314 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.167199    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ac21262d-9d0c-42cf-ad35-50819dfdeeed-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.179863 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.179837    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac21262d-9d0c-42cf-ad35-50819dfdeeed-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.181095 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.181073    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxtzt\" (UniqueName: \"kubernetes.io/projected/ac21262d-9d0c-42cf-ad35-50819dfdeeed-kube-api-access-nxtzt\") pod \"istiod-openshift-gateway-5677ccb49b-gjwmr\" (UID: \"ac21262d-9d0c-42cf-ad35-50819dfdeeed\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.311716 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.311631    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:32.639846 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:51:32.639813    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac21262d_9d0c_42cf_ad35_50819dfdeeed.slice/crio-4d8d94f40f9de384e1612a84ead0489cc444468449b955b6c8a54a76cb711954 WatchSource:0}: Error finding container 4d8d94f40f9de384e1612a84ead0489cc444468449b955b6c8a54a76cb711954: Status 404 returned error can't find the container with id 4d8d94f40f9de384e1612a84ead0489cc444468449b955b6c8a54a76cb711954
Sep 16 18:51:32.641733 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.641709    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"]
Sep 16 18:51:32.778192 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:32.778145    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr" event={"ID":"ac21262d-9d0c-42cf-ad35-50819dfdeeed","Type":"ContainerStarted","Data":"4d8d94f40f9de384e1612a84ead0489cc444468449b955b6c8a54a76cb711954"}
Sep 16 18:51:34.970282 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:34.970236    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:51:34.970576 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:34.970315    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:51:35.800633 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:35.800592    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr" event={"ID":"ac21262d-9d0c-42cf-ad35-50819dfdeeed","Type":"ContainerStarted","Data":"e6349e19c758eb6139e645f9e85c0055096ddeae8f9e627e85016a18bc2bde19"}
Sep 16 18:51:35.800817 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:35.800739    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:35.852807 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:35.852752    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr" podStartSLOduration=2.52520764 podStartE2EDuration="4.852733858s" totalImagesPullingTime="2.327526218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:51:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:51:35.850462704 +0000 UTC m=+723.638026878" watchObservedRunningTime="2026-09-16 18:51:35.852733858 +0000 UTC m=+723.640298012"
Sep 16 18:51:36.807765 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:36.807733    2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-gjwmr container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 18:51:36.808144 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:36.807788    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr" podUID="ac21262d-9d0c-42cf-ad35-50819dfdeeed" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:51:37.529996 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.529956    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"]
Sep 16 18:51:37.533654 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.533630    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.566724 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.566698    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-c8v9z\""
Sep 16 18:51:37.618974 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.618938    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f181395c-866f-4dd2-90d4-c316e2d0bf39-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.619143 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.618989    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f181395c-866f-4dd2-90d4-c316e2d0bf39-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.619143 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.619015    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnpvx\" (UniqueName: \"kubernetes.io/projected/f181395c-866f-4dd2-90d4-c316e2d0bf39-kube-api-access-cnpvx\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.619143 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.619045    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f181395c-866f-4dd2-90d4-c316e2d0bf39-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.619143 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.619067    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f181395c-866f-4dd2-90d4-c316e2d0bf39-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.619299 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.619161    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f181395c-866f-4dd2-90d4-c316e2d0bf39-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.619299 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.619239    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f181395c-866f-4dd2-90d4-c316e2d0bf39-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.619299 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.619284    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f181395c-866f-4dd2-90d4-c316e2d0bf39-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.619391 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.619305    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f181395c-866f-4dd2-90d4-c316e2d0bf39-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.682643 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.682596    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"]
Sep 16 18:51:37.719937 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.719904    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f181395c-866f-4dd2-90d4-c316e2d0bf39-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.720094 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.719943    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f181395c-866f-4dd2-90d4-c316e2d0bf39-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.720094 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.719962    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f181395c-866f-4dd2-90d4-c316e2d0bf39-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.720196 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.720090    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f181395c-866f-4dd2-90d4-c316e2d0bf39-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.720266 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.720191    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f181395c-866f-4dd2-90d4-c316e2d0bf39-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.720266 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.720224    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnpvx\" (UniqueName: \"kubernetes.io/projected/f181395c-866f-4dd2-90d4-c316e2d0bf39-kube-api-access-cnpvx\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.720360 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.720279    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f181395c-866f-4dd2-90d4-c316e2d0bf39-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.720360 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.720314    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f181395c-866f-4dd2-90d4-c316e2d0bf39-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.720360 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.720321    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f181395c-866f-4dd2-90d4-c316e2d0bf39-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.720500 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.720381    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f181395c-866f-4dd2-90d4-c316e2d0bf39-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.720500 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.720401    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f181395c-866f-4dd2-90d4-c316e2d0bf39-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.720602 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.720524    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f181395c-866f-4dd2-90d4-c316e2d0bf39-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.720715 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.720696    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f181395c-866f-4dd2-90d4-c316e2d0bf39-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.720754 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.720721    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f181395c-866f-4dd2-90d4-c316e2d0bf39-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.722534 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.722509    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f181395c-866f-4dd2-90d4-c316e2d0bf39-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.722784 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.722763    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f181395c-866f-4dd2-90d4-c316e2d0bf39-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.744429 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.744403    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f181395c-866f-4dd2-90d4-c316e2d0bf39-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.754405 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.754372    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnpvx\" (UniqueName: \"kubernetes.io/projected/f181395c-866f-4dd2-90d4-c316e2d0bf39-kube-api-access-cnpvx\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl\" (UID: \"f181395c-866f-4dd2-90d4-c316e2d0bf39\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:37.844012 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:37.843914    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:38.128225 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:38.128194    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"]
Sep 16 18:51:38.129599 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:51:38.129572    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf181395c_866f_4dd2_90d4_c316e2d0bf39.slice/crio-7bcbefcfdd2ce91d431fbe6c286c2e916f63b78f1867ef09d7d3a44e4cd83ac9 WatchSource:0}: Error finding container 7bcbefcfdd2ce91d431fbe6c286c2e916f63b78f1867ef09d7d3a44e4cd83ac9: Status 404 returned error can't find the container with id 7bcbefcfdd2ce91d431fbe6c286c2e916f63b78f1867ef09d7d3a44e4cd83ac9
Sep 16 18:51:38.817645 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:38.817603    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl" event={"ID":"f181395c-866f-4dd2-90d4-c316e2d0bf39","Type":"ContainerStarted","Data":"7bcbefcfdd2ce91d431fbe6c286c2e916f63b78f1867ef09d7d3a44e4cd83ac9"}
Sep 16 18:51:39.807895 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:39.807854    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-gjwmr"
Sep 16 18:51:40.576206 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:40.576152    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:51:40.576308 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:40.576243    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:51:40.576308 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:40.576276    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:51:40.830456 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:40.830371    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl" event={"ID":"f181395c-866f-4dd2-90d4-c316e2d0bf39","Type":"ContainerStarted","Data":"f3ca1608abb3c4dd21b9c2bea77fcfc2a139c5a1b42679f3b69cc0569ba473fa"}
Sep 16 18:51:40.844749 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:40.844724    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:40.879127 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:40.879023    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl" podStartSLOduration=1.434778428 podStartE2EDuration="3.879001754s" totalImagesPullingTime="2.444223326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:51:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:51:40.878237524 +0000 UTC m=+728.665801699" watchObservedRunningTime="2026-09-16 18:51:40.879001754 +0000 UTC m=+728.666565911"
Sep 16 18:51:41.849230 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:41.849198    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:42.842981 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:42.842951    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:51:42.843970 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:51:42.843946    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl"
Sep 16 18:52:29.446765 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.446728    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt"]
Sep 16 18:52:29.450238 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.450220    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt"
Sep 16 18:52:29.452669 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.452647    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 18:52:29.453380 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.453363    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 18:52:29.453510 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.453366    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5tnch\""
Sep 16 18:52:29.461519 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.461493    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt"]
Sep 16 18:52:29.539402 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.539364    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght"]
Sep 16 18:52:29.542991 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.542973    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght"
Sep 16 18:52:29.552849 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.552826    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght"]
Sep 16 18:52:29.592640 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.592607    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b9902250-670a-4cc6-9ff6-af699df92643-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt\" (UID: \"b9902250-670a-4cc6-9ff6-af699df92643\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt"
Sep 16 18:52:29.592819 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.592649    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b9902250-670a-4cc6-9ff6-af699df92643-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt\" (UID: \"b9902250-670a-4cc6-9ff6-af699df92643\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt"
Sep 16 18:52:29.592819 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.592683    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b4jl\" (UniqueName: \"kubernetes.io/projected/b9902250-670a-4cc6-9ff6-af699df92643-kube-api-access-2b4jl\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt\" (UID: \"b9902250-670a-4cc6-9ff6-af699df92643\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt"
Sep 16 18:52:29.663529 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.663496    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj"]
Sep 16 18:52:29.666998 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.666981    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj"
Sep 16 18:52:29.676850 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.676829    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj"]
Sep 16 18:52:29.693759 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.693730    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b41cfb60-0fb6-4a7c-954b-05011b0605fa-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght\" (UID: \"b41cfb60-0fb6-4a7c-954b-05011b0605fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght"
Sep 16 18:52:29.693917 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.693768    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b9902250-670a-4cc6-9ff6-af699df92643-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt\" (UID: \"b9902250-670a-4cc6-9ff6-af699df92643\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt"
Sep 16 18:52:29.693917 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.693854    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b4jl\" (UniqueName: \"kubernetes.io/projected/b9902250-670a-4cc6-9ff6-af699df92643-kube-api-access-2b4jl\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt\" (UID: \"b9902250-670a-4cc6-9ff6-af699df92643\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt"
Sep 16 18:52:29.693917 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.693903    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b41cfb60-0fb6-4a7c-954b-05011b0605fa-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght\" (UID: \"b41cfb60-0fb6-4a7c-954b-05011b0605fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght"
Sep 16 18:52:29.694065 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.693987    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th6k9\" (UniqueName: \"kubernetes.io/projected/b41cfb60-0fb6-4a7c-954b-05011b0605fa-kube-api-access-th6k9\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght\" (UID: \"b41cfb60-0fb6-4a7c-954b-05011b0605fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght"
Sep 16 18:52:29.694065 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.694038    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b9902250-670a-4cc6-9ff6-af699df92643-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt\" (UID: \"b9902250-670a-4cc6-9ff6-af699df92643\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt"
Sep 16 18:52:29.694164 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.694130    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b9902250-670a-4cc6-9ff6-af699df92643-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt\" (UID: \"b9902250-670a-4cc6-9ff6-af699df92643\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt"
Sep 16 18:52:29.694307 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.694287    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b9902250-670a-4cc6-9ff6-af699df92643-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt\" (UID: \"b9902250-670a-4cc6-9ff6-af699df92643\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt"
Sep 16 18:52:29.713978 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.713912    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b4jl\" (UniqueName: \"kubernetes.io/projected/b9902250-670a-4cc6-9ff6-af699df92643-kube-api-access-2b4jl\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt\" (UID: \"b9902250-670a-4cc6-9ff6-af699df92643\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt"
Sep 16 18:52:29.761903 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.761869    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt"
Sep 16 18:52:29.771238 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.771206    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p"]
Sep 16 18:52:29.775125 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.775102    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p"
Sep 16 18:52:29.787400 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.787352    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p"]
Sep 16 18:52:29.795254 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.795228    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b41cfb60-0fb6-4a7c-954b-05011b0605fa-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght\" (UID: \"b41cfb60-0fb6-4a7c-954b-05011b0605fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght"
Sep 16 18:52:29.795254 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.795263    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f547d787-a9de-4346-9d0e-3960835d3381-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj\" (UID: \"f547d787-a9de-4346-9d0e-3960835d3381\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj"
Sep 16 18:52:29.795473 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.795335    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f547d787-a9de-4346-9d0e-3960835d3381-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj\" (UID: \"f547d787-a9de-4346-9d0e-3960835d3381\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj"
Sep 16 18:52:29.795473 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.795371    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b41cfb60-0fb6-4a7c-954b-05011b0605fa-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght\" (UID: \"b41cfb60-0fb6-4a7c-954b-05011b0605fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght"
Sep 16 18:52:29.795473 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.795418    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4r7v\" (UniqueName: \"kubernetes.io/projected/f547d787-a9de-4346-9d0e-3960835d3381-kube-api-access-p4r7v\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj\" (UID: \"f547d787-a9de-4346-9d0e-3960835d3381\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj"
Sep 16 18:52:29.795638 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.795478    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th6k9\" (UniqueName: \"kubernetes.io/projected/b41cfb60-0fb6-4a7c-954b-05011b0605fa-kube-api-access-th6k9\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght\" (UID: \"b41cfb60-0fb6-4a7c-954b-05011b0605fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght"
Sep 16 18:52:29.795696 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.795645    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b41cfb60-0fb6-4a7c-954b-05011b0605fa-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght\" (UID: \"b41cfb60-0fb6-4a7c-954b-05011b0605fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght"
Sep 16 18:52:29.795933 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.795912    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b41cfb60-0fb6-4a7c-954b-05011b0605fa-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght\" (UID: \"b41cfb60-0fb6-4a7c-954b-05011b0605fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght"
Sep 16 18:52:29.805565 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.805505    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-th6k9\" (UniqueName: \"kubernetes.io/projected/b41cfb60-0fb6-4a7c-954b-05011b0605fa-kube-api-access-th6k9\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght\" (UID: \"b41cfb60-0fb6-4a7c-954b-05011b0605fa\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght"
Sep 16 18:52:29.852624 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.852592    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght"
Sep 16 18:52:29.896326 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.896296    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a2be90f-5c19-465e-a903-a611aad488cf-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p\" (UID: \"8a2be90f-5c19-465e-a903-a611aad488cf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p"
Sep 16 18:52:29.896489 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.896348    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f547d787-a9de-4346-9d0e-3960835d3381-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj\" (UID: \"f547d787-a9de-4346-9d0e-3960835d3381\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj"
Sep 16 18:52:29.896489 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.896368    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a2be90f-5c19-465e-a903-a611aad488cf-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p\" (UID: \"8a2be90f-5c19-465e-a903-a611aad488cf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p"
Sep 16 18:52:29.896489 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.896417    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f547d787-a9de-4346-9d0e-3960835d3381-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj\" (UID: \"f547d787-a9de-4346-9d0e-3960835d3381\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj"
Sep 16 18:52:29.896489 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.896450    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4r7v\" (UniqueName: \"kubernetes.io/projected/f547d787-a9de-4346-9d0e-3960835d3381-kube-api-access-p4r7v\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj\" (UID: \"f547d787-a9de-4346-9d0e-3960835d3381\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj"
Sep 16 18:52:29.896489 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.896466    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn9sw\" (UniqueName: \"kubernetes.io/projected/8a2be90f-5c19-465e-a903-a611aad488cf-kube-api-access-rn9sw\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p\" (UID: \"8a2be90f-5c19-465e-a903-a611aad488cf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p"
Sep 16 18:52:29.896852 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.896828    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f547d787-a9de-4346-9d0e-3960835d3381-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj\" (UID: \"f547d787-a9de-4346-9d0e-3960835d3381\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj"
Sep 16 18:52:29.896993 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.896969    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f547d787-a9de-4346-9d0e-3960835d3381-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj\" (UID: \"f547d787-a9de-4346-9d0e-3960835d3381\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj"
Sep 16 18:52:29.913121 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.913089    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4r7v\" (UniqueName: \"kubernetes.io/projected/f547d787-a9de-4346-9d0e-3960835d3381-kube-api-access-p4r7v\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj\" (UID: \"f547d787-a9de-4346-9d0e-3960835d3381\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj"
Sep 16 18:52:29.976135 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.976055    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj"
Sep 16 18:52:29.997324 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.997292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn9sw\" (UniqueName: \"kubernetes.io/projected/8a2be90f-5c19-465e-a903-a611aad488cf-kube-api-access-rn9sw\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p\" (UID: \"8a2be90f-5c19-465e-a903-a611aad488cf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p"
Sep 16 18:52:29.997492 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.997347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a2be90f-5c19-465e-a903-a611aad488cf-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p\" (UID: \"8a2be90f-5c19-465e-a903-a611aad488cf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p"
Sep 16 18:52:29.997492 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.997377    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a2be90f-5c19-465e-a903-a611aad488cf-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p\" (UID: \"8a2be90f-5c19-465e-a903-a611aad488cf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p"
Sep 16 18:52:29.997815 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.997793    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a2be90f-5c19-465e-a903-a611aad488cf-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p\" (UID: \"8a2be90f-5c19-465e-a903-a611aad488cf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p"
Sep 16 18:52:29.997868 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:29.997805    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a2be90f-5c19-465e-a903-a611aad488cf-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p\" (UID: \"8a2be90f-5c19-465e-a903-a611aad488cf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p"
Sep 16 18:52:30.009072 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:30.009039    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn9sw\" (UniqueName: \"kubernetes.io/projected/8a2be90f-5c19-465e-a903-a611aad488cf-kube-api-access-rn9sw\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p\" (UID: \"8a2be90f-5c19-465e-a903-a611aad488cf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p"
Sep 16 18:52:30.055273 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:30.055247    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt"]
Sep 16 18:52:30.058325 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:52:30.058297    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9902250_670a_4cc6_9ff6_af699df92643.slice/crio-4da8e09ee5dd2307cf12d876ff350108690c4e5d5db351f4128e7cd6af0bdd29 WatchSource:0}: Error finding container 4da8e09ee5dd2307cf12d876ff350108690c4e5d5db351f4128e7cd6af0bdd29: Status 404 returned error can't find the container with id 4da8e09ee5dd2307cf12d876ff350108690c4e5d5db351f4128e7cd6af0bdd29
Sep 16 18:52:30.077983 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:30.077957    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt" event={"ID":"b9902250-670a-4cc6-9ff6-af699df92643","Type":"ContainerStarted","Data":"4da8e09ee5dd2307cf12d876ff350108690c4e5d5db351f4128e7cd6af0bdd29"}
Sep 16 18:52:30.098855 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:30.098829    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p"
Sep 16 18:52:30.144509 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:30.144480    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght"]
Sep 16 18:52:30.145315 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:52:30.145288    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb41cfb60_0fb6_4a7c_954b_05011b0605fa.slice/crio-ff4eed54741e0095dc8649af443933a76f390cc338f77c8d1bdf4432421474e8 WatchSource:0}: Error finding container ff4eed54741e0095dc8649af443933a76f390cc338f77c8d1bdf4432421474e8: Status 404 returned error can't find the container with id ff4eed54741e0095dc8649af443933a76f390cc338f77c8d1bdf4432421474e8
Sep 16 18:52:30.259734 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:30.259709    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj"]
Sep 16 18:52:30.261583 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:52:30.261554    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf547d787_a9de_4346_9d0e_3960835d3381.slice/crio-7df2f3efb1c0e1806282fb01dba2843da6418024f008353d44c30fcddfeef585 WatchSource:0}: Error finding container 7df2f3efb1c0e1806282fb01dba2843da6418024f008353d44c30fcddfeef585: Status 404 returned error can't find the container with id 7df2f3efb1c0e1806282fb01dba2843da6418024f008353d44c30fcddfeef585
Sep 16 18:52:30.385946 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:30.385916    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p"]
Sep 16 18:52:30.413142 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:52:30.413109    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a2be90f_5c19_465e_a903_a611aad488cf.slice/crio-209ee69857113371a7b19c97bb1887d7262f71c868eff581ca76a41e4a994546 WatchSource:0}: Error finding container 209ee69857113371a7b19c97bb1887d7262f71c868eff581ca76a41e4a994546: Status 404 returned error can't find the container with id 209ee69857113371a7b19c97bb1887d7262f71c868eff581ca76a41e4a994546
Sep 16 18:52:31.084919 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:31.084891    2578 generic.go:356] "Generic (PLEG): container finished" podID="b41cfb60-0fb6-4a7c-954b-05011b0605fa" containerID="7690eab328ac04c6a7341eea4d471adcc5322baf4da887e8225e906de638aadc" exitCode=0
Sep 16 18:52:31.085385 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:31.084952    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght" event={"ID":"b41cfb60-0fb6-4a7c-954b-05011b0605fa","Type":"ContainerDied","Data":"7690eab328ac04c6a7341eea4d471adcc5322baf4da887e8225e906de638aadc"}
Sep 16 18:52:31.085385 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:31.085002    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght" event={"ID":"b41cfb60-0fb6-4a7c-954b-05011b0605fa","Type":"ContainerStarted","Data":"ff4eed54741e0095dc8649af443933a76f390cc338f77c8d1bdf4432421474e8"}
Sep 16 18:52:31.086814 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:31.086783    2578 generic.go:356] "Generic (PLEG): container finished" podID="b9902250-670a-4cc6-9ff6-af699df92643" containerID="e8859863f681d44b7114f9062e55c50fdf81bef99755cd09e88e88ed025812a5" exitCode=0
Sep 16 18:52:31.086921 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:31.086861    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt" event={"ID":"b9902250-670a-4cc6-9ff6-af699df92643","Type":"ContainerDied","Data":"e8859863f681d44b7114f9062e55c50fdf81bef99755cd09e88e88ed025812a5"}
Sep 16 18:52:31.088549 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:31.088528    2578 generic.go:356] "Generic (PLEG): container finished" podID="f547d787-a9de-4346-9d0e-3960835d3381" containerID="34d956613078c078638b5665ec05db0030d70307df6f414e4a264fdc2c326679" exitCode=0
Sep 16 18:52:31.088651 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:31.088626    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj" event={"ID":"f547d787-a9de-4346-9d0e-3960835d3381","Type":"ContainerDied","Data":"34d956613078c078638b5665ec05db0030d70307df6f414e4a264fdc2c326679"}
Sep 16 18:52:31.088735 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:31.088675    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj" event={"ID":"f547d787-a9de-4346-9d0e-3960835d3381","Type":"ContainerStarted","Data":"7df2f3efb1c0e1806282fb01dba2843da6418024f008353d44c30fcddfeef585"}
Sep 16 18:52:31.090390 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:31.090373    2578 generic.go:356] "Generic (PLEG): container finished" podID="8a2be90f-5c19-465e-a903-a611aad488cf" containerID="38a1f0a12534c34a9b47e9af2c16981d18954209407f82cb5d13706b31a6ebe3" exitCode=0
Sep 16 18:52:31.090491 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:31.090433    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p" event={"ID":"8a2be90f-5c19-465e-a903-a611aad488cf","Type":"ContainerDied","Data":"38a1f0a12534c34a9b47e9af2c16981d18954209407f82cb5d13706b31a6ebe3"}
Sep 16 18:52:31.090491 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:31.090461    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p" event={"ID":"8a2be90f-5c19-465e-a903-a611aad488cf","Type":"ContainerStarted","Data":"209ee69857113371a7b19c97bb1887d7262f71c868eff581ca76a41e4a994546"}
Sep 16 18:52:32.097911 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:32.097876    2578 generic.go:356] "Generic (PLEG): container finished" podID="b41cfb60-0fb6-4a7c-954b-05011b0605fa" containerID="ac27f0c2ca208f641d2867449d98f59bf988efbdc4b6c4bfa973d15352c381c7" exitCode=0
Sep 16 18:52:32.098357 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:32.097959    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght" event={"ID":"b41cfb60-0fb6-4a7c-954b-05011b0605fa","Type":"ContainerDied","Data":"ac27f0c2ca208f641d2867449d98f59bf988efbdc4b6c4bfa973d15352c381c7"}
Sep 16 18:52:32.100190 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:32.100148    2578 generic.go:356] "Generic (PLEG): container finished" podID="b9902250-670a-4cc6-9ff6-af699df92643" containerID="d0fee5d0554d9ecfc57616be192f47f8cc8fc947daa62a5c5ea48e1f0910e2ae" exitCode=0
Sep 16 18:52:32.107503 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:32.100615    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt" event={"ID":"b9902250-670a-4cc6-9ff6-af699df92643","Type":"ContainerDied","Data":"d0fee5d0554d9ecfc57616be192f47f8cc8fc947daa62a5c5ea48e1f0910e2ae"}
Sep 16 18:52:32.108466 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:32.108438    2578 generic.go:356] "Generic (PLEG): container finished" podID="8a2be90f-5c19-465e-a903-a611aad488cf" containerID="c0551b1c60e94cd66509f9ffc61a52ca1b46bf9f31e31d00799d971b65fc0f0d" exitCode=0
Sep 16 18:52:32.108585 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:32.108506    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p" event={"ID":"8a2be90f-5c19-465e-a903-a611aad488cf","Type":"ContainerDied","Data":"c0551b1c60e94cd66509f9ffc61a52ca1b46bf9f31e31d00799d971b65fc0f0d"}
Sep 16 18:52:33.115606 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:33.115572    2578 generic.go:356] "Generic (PLEG): container finished" podID="b41cfb60-0fb6-4a7c-954b-05011b0605fa" containerID="6deedece7ac81027175c0ee0e068cb0f6f7cd489a7d7a65ae900b40636fe1bb8" exitCode=0
Sep 16 18:52:33.116018 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:33.115663    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght" event={"ID":"b41cfb60-0fb6-4a7c-954b-05011b0605fa","Type":"ContainerDied","Data":"6deedece7ac81027175c0ee0e068cb0f6f7cd489a7d7a65ae900b40636fe1bb8"}
Sep 16 18:52:33.117969 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:33.117951    2578 generic.go:356] "Generic (PLEG): container finished" podID="b9902250-670a-4cc6-9ff6-af699df92643" containerID="9c03e9547debd8efad2a6f38c3abe20f22b384fe546c5fadde75cc0bceb09eac" exitCode=0
Sep 16 18:52:33.118049 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:33.118029    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt" event={"ID":"b9902250-670a-4cc6-9ff6-af699df92643","Type":"ContainerDied","Data":"9c03e9547debd8efad2a6f38c3abe20f22b384fe546c5fadde75cc0bceb09eac"}
Sep 16 18:52:33.120227 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:33.120204    2578 generic.go:356] "Generic (PLEG): container finished" podID="8a2be90f-5c19-465e-a903-a611aad488cf" containerID="6b2d3921fc021b2c677228338cf5093f13227099c9d856074c2f69d54f02d3cd" exitCode=0
Sep 16 18:52:33.120340 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:33.120234    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p" event={"ID":"8a2be90f-5c19-465e-a903-a611aad488cf","Type":"ContainerDied","Data":"6b2d3921fc021b2c677228338cf5093f13227099c9d856074c2f69d54f02d3cd"}
Sep 16 18:52:34.280112 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.280083    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p"
Sep 16 18:52:34.321294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.321268    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt"
Sep 16 18:52:34.325551 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.325524    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght"
Sep 16 18:52:34.436935 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.436840    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b9902250-670a-4cc6-9ff6-af699df92643-bundle\") pod \"b9902250-670a-4cc6-9ff6-af699df92643\" (UID: \"b9902250-670a-4cc6-9ff6-af699df92643\") "
Sep 16 18:52:34.436935 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.436889    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b41cfb60-0fb6-4a7c-954b-05011b0605fa-util\") pod \"b41cfb60-0fb6-4a7c-954b-05011b0605fa\" (UID: \"b41cfb60-0fb6-4a7c-954b-05011b0605fa\") "
Sep 16 18:52:34.436935 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.436910    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rn9sw\" (UniqueName: \"kubernetes.io/projected/8a2be90f-5c19-465e-a903-a611aad488cf-kube-api-access-rn9sw\") pod \"8a2be90f-5c19-465e-a903-a611aad488cf\" (UID: \"8a2be90f-5c19-465e-a903-a611aad488cf\") "
Sep 16 18:52:34.437249 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.436959    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a2be90f-5c19-465e-a903-a611aad488cf-bundle\") pod \"8a2be90f-5c19-465e-a903-a611aad488cf\" (UID: \"8a2be90f-5c19-465e-a903-a611aad488cf\") "
Sep 16 18:52:34.437249 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.437030    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a2be90f-5c19-465e-a903-a611aad488cf-util\") pod \"8a2be90f-5c19-465e-a903-a611aad488cf\" (UID: \"8a2be90f-5c19-465e-a903-a611aad488cf\") "
Sep 16 18:52:34.437249 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.437088    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b9902250-670a-4cc6-9ff6-af699df92643-util\") pod \"b9902250-670a-4cc6-9ff6-af699df92643\" (UID: \"b9902250-670a-4cc6-9ff6-af699df92643\") "
Sep 16 18:52:34.437249 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.437117    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2b4jl\" (UniqueName: \"kubernetes.io/projected/b9902250-670a-4cc6-9ff6-af699df92643-kube-api-access-2b4jl\") pod \"b9902250-670a-4cc6-9ff6-af699df92643\" (UID: \"b9902250-670a-4cc6-9ff6-af699df92643\") "
Sep 16 18:52:34.437249 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.437145    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b41cfb60-0fb6-4a7c-954b-05011b0605fa-bundle\") pod \"b41cfb60-0fb6-4a7c-954b-05011b0605fa\" (UID: \"b41cfb60-0fb6-4a7c-954b-05011b0605fa\") "
Sep 16 18:52:34.437249 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.437208    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-th6k9\" (UniqueName: \"kubernetes.io/projected/b41cfb60-0fb6-4a7c-954b-05011b0605fa-kube-api-access-th6k9\") pod \"b41cfb60-0fb6-4a7c-954b-05011b0605fa\" (UID: \"b41cfb60-0fb6-4a7c-954b-05011b0605fa\") "
Sep 16 18:52:34.437546 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.437506    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9902250-670a-4cc6-9ff6-af699df92643-bundle" (OuterVolumeSpecName: "bundle") pod "b9902250-670a-4cc6-9ff6-af699df92643" (UID: "b9902250-670a-4cc6-9ff6-af699df92643"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:52:34.437913 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.437881    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a2be90f-5c19-465e-a903-a611aad488cf-bundle" (OuterVolumeSpecName: "bundle") pod "8a2be90f-5c19-465e-a903-a611aad488cf" (UID: "8a2be90f-5c19-465e-a903-a611aad488cf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:52:34.438192 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.438128    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b41cfb60-0fb6-4a7c-954b-05011b0605fa-bundle" (OuterVolumeSpecName: "bundle") pod "b41cfb60-0fb6-4a7c-954b-05011b0605fa" (UID: "b41cfb60-0fb6-4a7c-954b-05011b0605fa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:52:34.439803 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.439780    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a2be90f-5c19-465e-a903-a611aad488cf-kube-api-access-rn9sw" (OuterVolumeSpecName: "kube-api-access-rn9sw") pod "8a2be90f-5c19-465e-a903-a611aad488cf" (UID: "8a2be90f-5c19-465e-a903-a611aad488cf"). InnerVolumeSpecName "kube-api-access-rn9sw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:52:34.439881 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.439812    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b41cfb60-0fb6-4a7c-954b-05011b0605fa-kube-api-access-th6k9" (OuterVolumeSpecName: "kube-api-access-th6k9") pod "b41cfb60-0fb6-4a7c-954b-05011b0605fa" (UID: "b41cfb60-0fb6-4a7c-954b-05011b0605fa"). InnerVolumeSpecName "kube-api-access-th6k9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:52:34.440161 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.440140    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9902250-670a-4cc6-9ff6-af699df92643-kube-api-access-2b4jl" (OuterVolumeSpecName: "kube-api-access-2b4jl") pod "b9902250-670a-4cc6-9ff6-af699df92643" (UID: "b9902250-670a-4cc6-9ff6-af699df92643"). InnerVolumeSpecName "kube-api-access-2b4jl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:52:34.443780 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.443748    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a2be90f-5c19-465e-a903-a611aad488cf-util" (OuterVolumeSpecName: "util") pod "8a2be90f-5c19-465e-a903-a611aad488cf" (UID: "8a2be90f-5c19-465e-a903-a611aad488cf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:52:34.446110 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.446085    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b41cfb60-0fb6-4a7c-954b-05011b0605fa-util" (OuterVolumeSpecName: "util") pod "b41cfb60-0fb6-4a7c-954b-05011b0605fa" (UID: "b41cfb60-0fb6-4a7c-954b-05011b0605fa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:52:34.446373 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.446354    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9902250-670a-4cc6-9ff6-af699df92643-util" (OuterVolumeSpecName: "util") pod "b9902250-670a-4cc6-9ff6-af699df92643" (UID: "b9902250-670a-4cc6-9ff6-af699df92643"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:52:34.538709 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.538675    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8a2be90f-5c19-465e-a903-a611aad488cf-util\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:52:34.538709 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.538705    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b9902250-670a-4cc6-9ff6-af699df92643-util\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:52:34.538709 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.538715    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2b4jl\" (UniqueName: \"kubernetes.io/projected/b9902250-670a-4cc6-9ff6-af699df92643-kube-api-access-2b4jl\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:52:34.538953 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.538724    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b41cfb60-0fb6-4a7c-954b-05011b0605fa-bundle\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:52:34.538953 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.538733    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-th6k9\" (UniqueName: \"kubernetes.io/projected/b41cfb60-0fb6-4a7c-954b-05011b0605fa-kube-api-access-th6k9\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:52:34.538953 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.538742    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b9902250-670a-4cc6-9ff6-af699df92643-bundle\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:52:34.538953 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.538750    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b41cfb60-0fb6-4a7c-954b-05011b0605fa-util\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:52:34.538953 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.538757    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rn9sw\" (UniqueName: \"kubernetes.io/projected/8a2be90f-5c19-465e-a903-a611aad488cf-kube-api-access-rn9sw\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:52:34.538953 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:34.538765    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8a2be90f-5c19-465e-a903-a611aad488cf-bundle\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:52:35.132166 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:35.132141    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght"
Sep 16 18:52:35.132359 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:35.132164    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c6cght" event={"ID":"b41cfb60-0fb6-4a7c-954b-05011b0605fa","Type":"ContainerDied","Data":"ff4eed54741e0095dc8649af443933a76f390cc338f77c8d1bdf4432421474e8"}
Sep 16 18:52:35.132359 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:35.132243    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff4eed54741e0095dc8649af443933a76f390cc338f77c8d1bdf4432421474e8"
Sep 16 18:52:35.134481 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:35.134460    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt"
Sep 16 18:52:35.134613 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:35.134462    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d72wnqt" event={"ID":"b9902250-670a-4cc6-9ff6-af699df92643","Type":"ContainerDied","Data":"4da8e09ee5dd2307cf12d876ff350108690c4e5d5db351f4128e7cd6af0bdd29"}
Sep 16 18:52:35.134613 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:35.134581    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4da8e09ee5dd2307cf12d876ff350108690c4e5d5db351f4128e7cd6af0bdd29"
Sep 16 18:52:35.136517 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:35.136494    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p" event={"ID":"8a2be90f-5c19-465e-a903-a611aad488cf","Type":"ContainerDied","Data":"209ee69857113371a7b19c97bb1887d7262f71c868eff581ca76a41e4a994546"}
Sep 16 18:52:35.136621 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:35.136521    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c646f8p"
Sep 16 18:52:35.136621 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:35.136524    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="209ee69857113371a7b19c97bb1887d7262f71c868eff581ca76a41e4a994546"
Sep 16 18:52:41.169008 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:41.168974    2578 generic.go:356] "Generic (PLEG): container finished" podID="f547d787-a9de-4346-9d0e-3960835d3381" containerID="782ff605d295cd284386be095ade5cf2b123b2f91296a4416454bed4a9595be5" exitCode=0
Sep 16 18:52:41.169400 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:41.169069    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj" event={"ID":"f547d787-a9de-4346-9d0e-3960835d3381","Type":"ContainerDied","Data":"782ff605d295cd284386be095ade5cf2b123b2f91296a4416454bed4a9595be5"}
Sep 16 18:52:42.176895 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:42.176861    2578 generic.go:356] "Generic (PLEG): container finished" podID="f547d787-a9de-4346-9d0e-3960835d3381" containerID="da158c468be835798720a44a079fdb73f5933e48ea6b8acac1b67d434b476e43" exitCode=0
Sep 16 18:52:42.177297 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:42.176911    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj" event={"ID":"f547d787-a9de-4346-9d0e-3960835d3381","Type":"ContainerDied","Data":"da158c468be835798720a44a079fdb73f5933e48ea6b8acac1b67d434b476e43"}
Sep 16 18:52:43.320511 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:43.320483    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj"
Sep 16 18:52:43.413623 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:43.413575    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f547d787-a9de-4346-9d0e-3960835d3381-util\") pod \"f547d787-a9de-4346-9d0e-3960835d3381\" (UID: \"f547d787-a9de-4346-9d0e-3960835d3381\") "
Sep 16 18:52:43.413623 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:43.413627    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4r7v\" (UniqueName: \"kubernetes.io/projected/f547d787-a9de-4346-9d0e-3960835d3381-kube-api-access-p4r7v\") pod \"f547d787-a9de-4346-9d0e-3960835d3381\" (UID: \"f547d787-a9de-4346-9d0e-3960835d3381\") "
Sep 16 18:52:43.413836 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:43.413672    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f547d787-a9de-4346-9d0e-3960835d3381-bundle\") pod \"f547d787-a9de-4346-9d0e-3960835d3381\" (UID: \"f547d787-a9de-4346-9d0e-3960835d3381\") "
Sep 16 18:52:43.414393 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:43.414357    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f547d787-a9de-4346-9d0e-3960835d3381-bundle" (OuterVolumeSpecName: "bundle") pod "f547d787-a9de-4346-9d0e-3960835d3381" (UID: "f547d787-a9de-4346-9d0e-3960835d3381"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:52:43.415782 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:43.415751    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f547d787-a9de-4346-9d0e-3960835d3381-kube-api-access-p4r7v" (OuterVolumeSpecName: "kube-api-access-p4r7v") pod "f547d787-a9de-4346-9d0e-3960835d3381" (UID: "f547d787-a9de-4346-9d0e-3960835d3381"). InnerVolumeSpecName "kube-api-access-p4r7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:52:43.418077 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:43.418052    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f547d787-a9de-4346-9d0e-3960835d3381-util" (OuterVolumeSpecName: "util") pod "f547d787-a9de-4346-9d0e-3960835d3381" (UID: "f547d787-a9de-4346-9d0e-3960835d3381"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:52:43.514979 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:43.514880    2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f547d787-a9de-4346-9d0e-3960835d3381-util\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:52:43.514979 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:43.514924    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4r7v\" (UniqueName: \"kubernetes.io/projected/f547d787-a9de-4346-9d0e-3960835d3381-kube-api-access-p4r7v\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:52:43.514979 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:43.514939    2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f547d787-a9de-4346-9d0e-3960835d3381-bundle\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:52:44.191090 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:44.191056    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj" event={"ID":"f547d787-a9de-4346-9d0e-3960835d3381","Type":"ContainerDied","Data":"7df2f3efb1c0e1806282fb01dba2843da6418024f008353d44c30fcddfeef585"}
Sep 16 18:52:44.191258 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:44.191102    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7df2f3efb1c0e1806282fb01dba2843da6418024f008353d44c30fcddfeef585"
Sep 16 18:52:44.191258 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:44.191066    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ah6rgj"
Sep 16 18:52:53.093260 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093221    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-tfkdv"]
Sep 16 18:52:53.093649 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093632    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b41cfb60-0fb6-4a7c-954b-05011b0605fa" containerName="util"
Sep 16 18:52:53.093706 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093651    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b41cfb60-0fb6-4a7c-954b-05011b0605fa" containerName="util"
Sep 16 18:52:53.093706 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093662    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a2be90f-5c19-465e-a903-a611aad488cf" containerName="pull"
Sep 16 18:52:53.093706 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093669    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a2be90f-5c19-465e-a903-a611aad488cf" containerName="pull"
Sep 16 18:52:53.093706 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093683    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b41cfb60-0fb6-4a7c-954b-05011b0605fa" containerName="pull"
Sep 16 18:52:53.093706 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093691    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b41cfb60-0fb6-4a7c-954b-05011b0605fa" containerName="pull"
Sep 16 18:52:53.093706 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093699    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a2be90f-5c19-465e-a903-a611aad488cf" containerName="extract"
Sep 16 18:52:53.093706 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093704    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a2be90f-5c19-465e-a903-a611aad488cf" containerName="extract"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093710    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f547d787-a9de-4346-9d0e-3960835d3381" containerName="util"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093715    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f547d787-a9de-4346-9d0e-3960835d3381" containerName="util"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093722    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9902250-670a-4cc6-9ff6-af699df92643" containerName="extract"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093726    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9902250-670a-4cc6-9ff6-af699df92643" containerName="extract"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093734    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9902250-670a-4cc6-9ff6-af699df92643" containerName="util"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093739    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9902250-670a-4cc6-9ff6-af699df92643" containerName="util"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093746    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a2be90f-5c19-465e-a903-a611aad488cf" containerName="util"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093753    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a2be90f-5c19-465e-a903-a611aad488cf" containerName="util"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093766    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f547d787-a9de-4346-9d0e-3960835d3381" containerName="extract"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093771    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f547d787-a9de-4346-9d0e-3960835d3381" containerName="extract"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093779    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9902250-670a-4cc6-9ff6-af699df92643" containerName="pull"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093783    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9902250-670a-4cc6-9ff6-af699df92643" containerName="pull"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093791    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f547d787-a9de-4346-9d0e-3960835d3381" containerName="pull"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093796    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f547d787-a9de-4346-9d0e-3960835d3381" containerName="pull"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093802    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b41cfb60-0fb6-4a7c-954b-05011b0605fa" containerName="extract"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093808    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b41cfb60-0fb6-4a7c-954b-05011b0605fa" containerName="extract"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093885    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b41cfb60-0fb6-4a7c-954b-05011b0605fa" containerName="extract"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093894    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f547d787-a9de-4346-9d0e-3960835d3381" containerName="extract"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093902    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9902250-670a-4cc6-9ff6-af699df92643" containerName="extract"
Sep 16 18:52:53.093905 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.093909    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a2be90f-5c19-465e-a903-a611aad488cf" containerName="extract"
Sep 16 18:52:53.097307 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.097291    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-tfkdv"
Sep 16 18:52:53.100274 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.100242    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 16 18:52:53.100399 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.100330    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 16 18:52:53.100851 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.100831    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-5wmtf\""
Sep 16 18:52:53.115805 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.115777    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-tfkdv"]
Sep 16 18:52:53.200801 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.200766    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwb7w\" (UniqueName: \"kubernetes.io/projected/7e508345-cf51-4288-a312-79df0e23fcbf-kube-api-access-kwb7w\") pod \"authorino-operator-76b478bd78-tfkdv\" (UID: \"7e508345-cf51-4288-a312-79df0e23fcbf\") " pod="kuadrant-system/authorino-operator-76b478bd78-tfkdv"
Sep 16 18:52:53.301714 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.301678    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwb7w\" (UniqueName: \"kubernetes.io/projected/7e508345-cf51-4288-a312-79df0e23fcbf-kube-api-access-kwb7w\") pod \"authorino-operator-76b478bd78-tfkdv\" (UID: \"7e508345-cf51-4288-a312-79df0e23fcbf\") " pod="kuadrant-system/authorino-operator-76b478bd78-tfkdv"
Sep 16 18:52:53.311165 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.311141    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwb7w\" (UniqueName: \"kubernetes.io/projected/7e508345-cf51-4288-a312-79df0e23fcbf-kube-api-access-kwb7w\") pod \"authorino-operator-76b478bd78-tfkdv\" (UID: \"7e508345-cf51-4288-a312-79df0e23fcbf\") " pod="kuadrant-system/authorino-operator-76b478bd78-tfkdv"
Sep 16 18:52:53.407615 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.407579    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-tfkdv"
Sep 16 18:52:53.683099 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:53.683073    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-tfkdv"]
Sep 16 18:52:53.684786 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:52:53.684750    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e508345_cf51_4288_a312_79df0e23fcbf.slice/crio-835b1e5cca47d9dfaae9bb46c8b95436f69419f4987cdf632dd5834e02e78bf2 WatchSource:0}: Error finding container 835b1e5cca47d9dfaae9bb46c8b95436f69419f4987cdf632dd5834e02e78bf2: Status 404 returned error can't find the container with id 835b1e5cca47d9dfaae9bb46c8b95436f69419f4987cdf632dd5834e02e78bf2
Sep 16 18:52:54.022045 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:54.021962    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5747b7df4-z87km"]
Sep 16 18:52:54.245088 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:54.245053    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-tfkdv" event={"ID":"7e508345-cf51-4288-a312-79df0e23fcbf","Type":"ContainerStarted","Data":"835b1e5cca47d9dfaae9bb46c8b95436f69419f4987cdf632dd5834e02e78bf2"}
Sep 16 18:52:56.259954 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:56.259912    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-tfkdv" event={"ID":"7e508345-cf51-4288-a312-79df0e23fcbf","Type":"ContainerStarted","Data":"8bdac278687a8f367bbd4711a6bff76a4d58518113f053ff557379b83bcf7a15"}
Sep 16 18:52:56.260405 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:56.260056    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-tfkdv"
Sep 16 18:52:56.279923 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:56.279820    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-tfkdv" podStartSLOduration=0.943969964 podStartE2EDuration="3.279803401s" totalImagesPullingTime="2.335833437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:52:56.278149768 +0000 UTC m=+804.065713922" watchObservedRunningTime="2026-09-16 18:52:56.279803401 +0000 UTC m=+804.067367557"
Sep 16 18:52:57.338738 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:57.338695    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"]
Sep 16 18:52:57.342587 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:57.342564    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"
Sep 16 18:52:57.344993 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:57.344973    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-rxc4d\""
Sep 16 18:52:57.352863 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:57.352839    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"]
Sep 16 18:52:57.439463 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:57.439418    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt4lz\" (UniqueName: \"kubernetes.io/projected/7c4285eb-2b53-4079-bddf-8fbfdd1364c6-kube-api-access-kt4lz\") pod \"limitador-operator-controller-manager-74495c69f8-ctdlj\" (UID: \"7c4285eb-2b53-4079-bddf-8fbfdd1364c6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"
Sep 16 18:52:57.540378 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:57.540334    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt4lz\" (UniqueName: \"kubernetes.io/projected/7c4285eb-2b53-4079-bddf-8fbfdd1364c6-kube-api-access-kt4lz\") pod \"limitador-operator-controller-manager-74495c69f8-ctdlj\" (UID: \"7c4285eb-2b53-4079-bddf-8fbfdd1364c6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"
Sep 16 18:52:57.549229 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:57.549202    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt4lz\" (UniqueName: \"kubernetes.io/projected/7c4285eb-2b53-4079-bddf-8fbfdd1364c6-kube-api-access-kt4lz\") pod \"limitador-operator-controller-manager-74495c69f8-ctdlj\" (UID: \"7c4285eb-2b53-4079-bddf-8fbfdd1364c6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"
Sep 16 18:52:57.652781 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:57.652742    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"
Sep 16 18:52:57.943477 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:57.943450    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"]
Sep 16 18:52:57.945765 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:52:57.945736    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c4285eb_2b53_4079_bddf_8fbfdd1364c6.slice/crio-dbb9f39d72bd9e60fc049f8dcfec9d45f2adf4ba760600fc4c11c845c136abee WatchSource:0}: Error finding container dbb9f39d72bd9e60fc049f8dcfec9d45f2adf4ba760600fc4c11c845c136abee: Status 404 returned error can't find the container with id dbb9f39d72bd9e60fc049f8dcfec9d45f2adf4ba760600fc4c11c845c136abee
Sep 16 18:52:58.271716 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:52:58.271626    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj" event={"ID":"7c4285eb-2b53-4079-bddf-8fbfdd1364c6","Type":"ContainerStarted","Data":"dbb9f39d72bd9e60fc049f8dcfec9d45f2adf4ba760600fc4c11c845c136abee"}
Sep 16 18:53:00.284590 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:00.284550    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj" event={"ID":"7c4285eb-2b53-4079-bddf-8fbfdd1364c6","Type":"ContainerStarted","Data":"3ad9b4cfcfd043422624b32eb749126e57a700bac9d8f0943cfe8b92d9a35cbd"}
Sep 16 18:53:00.284979 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:00.284648    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"
Sep 16 18:53:00.309402 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:00.309351    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj" podStartSLOduration=1.7569435599999998 podStartE2EDuration="3.309333066s" totalImagesPullingTime="1.552389506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:52:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:53:00.307390959 +0000 UTC m=+808.094955134" watchObservedRunningTime="2026-09-16 18:53:00.309333066 +0000 UTC m=+808.096897220"
Sep 16 18:53:07.267312 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:07.267279    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-tfkdv"
Sep 16 18:53:08.117908 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:08.117870    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"]
Sep 16 18:53:08.118232 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:08.118159    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj" podUID="7c4285eb-2b53-4079-bddf-8fbfdd1364c6" containerName="manager" containerID="cri-o://3ad9b4cfcfd043422624b32eb749126e57a700bac9d8f0943cfe8b92d9a35cbd" gracePeriod=2
Sep 16 18:53:08.120124 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:08.120098    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"
Sep 16 18:53:08.148991 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:53:08.148941    2578 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"7c4285eb-2b53-4079-bddf-8fbfdd1364c6\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-ctdlj\": pods \"limitador-operator-controller-manager-74495c69f8-ctdlj\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"
Sep 16 18:53:08.150862 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:08.150816    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"]
Sep 16 18:53:08.150992 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:53:08.150913    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-ctdlj\" is forbidden: User \"system:node:ip-10-0-134-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-234.ec2.internal' and this object" podUID="7c4285eb-2b53-4079-bddf-8fbfdd1364c6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"
Sep 16 18:53:08.330858 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:08.330827    2578 generic.go:356] "Generic (PLEG): container finished" podID="7c4285eb-2b53-4079-bddf-8fbfdd1364c6" containerID="3ad9b4cfcfd043422624b32eb749126e57a700bac9d8f0943cfe8b92d9a35cbd" exitCode=0
Sep 16 18:53:08.360496 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:08.360475    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"
Sep 16 18:53:08.362451 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:53:08.362423    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-ctdlj\" is forbidden: User \"system:node:ip-10-0-134-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-234.ec2.internal' and this object" podUID="7c4285eb-2b53-4079-bddf-8fbfdd1364c6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"
Sep 16 18:53:08.442834 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:08.442742    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kt4lz\" (UniqueName: \"kubernetes.io/projected/7c4285eb-2b53-4079-bddf-8fbfdd1364c6-kube-api-access-kt4lz\") pod \"7c4285eb-2b53-4079-bddf-8fbfdd1364c6\" (UID: \"7c4285eb-2b53-4079-bddf-8fbfdd1364c6\") "
Sep 16 18:53:08.444939 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:08.444910    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c4285eb-2b53-4079-bddf-8fbfdd1364c6-kube-api-access-kt4lz" (OuterVolumeSpecName: "kube-api-access-kt4lz") pod "7c4285eb-2b53-4079-bddf-8fbfdd1364c6" (UID: "7c4285eb-2b53-4079-bddf-8fbfdd1364c6"). InnerVolumeSpecName "kube-api-access-kt4lz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:53:08.543566 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:08.543531    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kt4lz\" (UniqueName: \"kubernetes.io/projected/7c4285eb-2b53-4079-bddf-8fbfdd1364c6-kube-api-access-kt4lz\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:53:08.709906 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:08.709830    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c4285eb-2b53-4079-bddf-8fbfdd1364c6" path="/var/lib/kubelet/pods/7c4285eb-2b53-4079-bddf-8fbfdd1364c6/volumes"
Sep 16 18:53:09.162433 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.162397    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"]
Sep 16 18:53:09.162747 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.162736    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c4285eb-2b53-4079-bddf-8fbfdd1364c6" containerName="manager"
Sep 16 18:53:09.162792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.162748    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c4285eb-2b53-4079-bddf-8fbfdd1364c6" containerName="manager"
Sep 16 18:53:09.162826 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.162818    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c4285eb-2b53-4079-bddf-8fbfdd1364c6" containerName="manager"
Sep 16 18:53:09.165881 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.165864    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"
Sep 16 18:53:09.168130 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.168107    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8pkfh\""
Sep 16 18:53:09.177238 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.177216    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"]
Sep 16 18:53:09.251166 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.251128    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8098ba1b-391e-4688-98a2-f7ad0a7a72d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8msmr\" (UID: \"8098ba1b-391e-4688-98a2-f7ad0a7a72d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"
Sep 16 18:53:09.251166 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.251199    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv6f7\" (UniqueName: \"kubernetes.io/projected/8098ba1b-391e-4688-98a2-f7ad0a7a72d8-kube-api-access-rv6f7\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8msmr\" (UID: \"8098ba1b-391e-4688-98a2-f7ad0a7a72d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"
Sep 16 18:53:09.337236 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.337210    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ctdlj"
Sep 16 18:53:09.337694 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.337244    2578 scope.go:117] "RemoveContainer" containerID="3ad9b4cfcfd043422624b32eb749126e57a700bac9d8f0943cfe8b92d9a35cbd"
Sep 16 18:53:09.351951 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.351921    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8098ba1b-391e-4688-98a2-f7ad0a7a72d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8msmr\" (UID: \"8098ba1b-391e-4688-98a2-f7ad0a7a72d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"
Sep 16 18:53:09.352084 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.351968    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv6f7\" (UniqueName: \"kubernetes.io/projected/8098ba1b-391e-4688-98a2-f7ad0a7a72d8-kube-api-access-rv6f7\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8msmr\" (UID: \"8098ba1b-391e-4688-98a2-f7ad0a7a72d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"
Sep 16 18:53:09.352346 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.352322    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8098ba1b-391e-4688-98a2-f7ad0a7a72d8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8msmr\" (UID: \"8098ba1b-391e-4688-98a2-f7ad0a7a72d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"
Sep 16 18:53:09.360622 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.360598    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv6f7\" (UniqueName: \"kubernetes.io/projected/8098ba1b-391e-4688-98a2-f7ad0a7a72d8-kube-api-access-rv6f7\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8msmr\" (UID: \"8098ba1b-391e-4688-98a2-f7ad0a7a72d8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"
Sep 16 18:53:09.476526 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.476443    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"
Sep 16 18:53:09.753314 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:09.753212    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"]
Sep 16 18:53:09.755818 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:53:09.755794    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8098ba1b_391e_4688_98a2_f7ad0a7a72d8.slice/crio-20310e42fe99dbfe7942d030795fe0d7dfcdd3695168979e2df591dd138b2a5b WatchSource:0}: Error finding container 20310e42fe99dbfe7942d030795fe0d7dfcdd3695168979e2df591dd138b2a5b: Status 404 returned error can't find the container with id 20310e42fe99dbfe7942d030795fe0d7dfcdd3695168979e2df591dd138b2a5b
Sep 16 18:53:10.343775 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:10.343737    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr" event={"ID":"8098ba1b-391e-4688-98a2-f7ad0a7a72d8","Type":"ContainerStarted","Data":"20310e42fe99dbfe7942d030795fe0d7dfcdd3695168979e2df591dd138b2a5b"}
Sep 16 18:53:16.386759 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:16.386716    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr" event={"ID":"8098ba1b-391e-4688-98a2-f7ad0a7a72d8","Type":"ContainerStarted","Data":"1e0ec0656d44329d55c323579124b2750e5e3eddf75bdb991892470caa5cf049"}
Sep 16 18:53:16.387146 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:16.386792    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"
Sep 16 18:53:16.410570 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:16.410512    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr" podStartSLOduration=1.666176262 podStartE2EDuration="7.410494359s" totalImagesPullingTime="5.744318097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:53:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:53:16.408308777 +0000 UTC m=+824.195872953" watchObservedRunningTime="2026-09-16 18:53:16.410494359 +0000 UTC m=+824.198058513"
Sep 16 18:53:19.041372 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.041332    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5747b7df4-z87km" podUID="17943503-a564-4ab8-b255-1173fe3f15ef" containerName="console" containerID="cri-o://79af1c1f1983453bfb95741d7d8e2b43a10d8f53f648bc18fef2e9e7b445b885" gracePeriod=15
Sep 16 18:53:19.291653 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.291591    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:53:19.404532 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.404501    2578 generic.go:356] "Generic (PLEG): container finished" podID="17943503-a564-4ab8-b255-1173fe3f15ef" containerID="79af1c1f1983453bfb95741d7d8e2b43a10d8f53f648bc18fef2e9e7b445b885" exitCode=0
Sep 16 18:53:19.404702 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.404544    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5747b7df4-z87km"
Sep 16 18:53:19.404702 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.404591    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5747b7df4-z87km" event={"ID":"17943503-a564-4ab8-b255-1173fe3f15ef","Type":"ContainerDied","Data":"79af1c1f1983453bfb95741d7d8e2b43a10d8f53f648bc18fef2e9e7b445b885"}
Sep 16 18:53:19.404702 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.404642    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5747b7df4-z87km" event={"ID":"17943503-a564-4ab8-b255-1173fe3f15ef","Type":"ContainerDied","Data":"b0673aadd3375e0dbff82dddb0b61093bf33b3dcc050030f2edc025018d946f4"}
Sep 16 18:53:19.404702 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.404662    2578 scope.go:117] "RemoveContainer" containerID="79af1c1f1983453bfb95741d7d8e2b43a10d8f53f648bc18fef2e9e7b445b885"
Sep 16 18:53:19.413486 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.413470    2578 scope.go:117] "RemoveContainer" containerID="79af1c1f1983453bfb95741d7d8e2b43a10d8f53f648bc18fef2e9e7b445b885"
Sep 16 18:53:19.413766 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:53:19.413749    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79af1c1f1983453bfb95741d7d8e2b43a10d8f53f648bc18fef2e9e7b445b885\": container with ID starting with 79af1c1f1983453bfb95741d7d8e2b43a10d8f53f648bc18fef2e9e7b445b885 not found: ID does not exist" containerID="79af1c1f1983453bfb95741d7d8e2b43a10d8f53f648bc18fef2e9e7b445b885"
Sep 16 18:53:19.413815 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.413775    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79af1c1f1983453bfb95741d7d8e2b43a10d8f53f648bc18fef2e9e7b445b885"} err="failed to get container status \"79af1c1f1983453bfb95741d7d8e2b43a10d8f53f648bc18fef2e9e7b445b885\": rpc error: code = NotFound desc = could not find container \"79af1c1f1983453bfb95741d7d8e2b43a10d8f53f648bc18fef2e9e7b445b885\": container with ID starting with 79af1c1f1983453bfb95741d7d8e2b43a10d8f53f648bc18fef2e9e7b445b885 not found: ID does not exist"
Sep 16 18:53:19.442623 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.442594    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-console-config\") pod \"17943503-a564-4ab8-b255-1173fe3f15ef\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") "
Sep 16 18:53:19.442751 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.442628    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh44z\" (UniqueName: \"kubernetes.io/projected/17943503-a564-4ab8-b255-1173fe3f15ef-kube-api-access-qh44z\") pod \"17943503-a564-4ab8-b255-1173fe3f15ef\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") "
Sep 16 18:53:19.442751 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.442672    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-service-ca\") pod \"17943503-a564-4ab8-b255-1173fe3f15ef\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") "
Sep 16 18:53:19.442751 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.442697    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-oauth-serving-cert\") pod \"17943503-a564-4ab8-b255-1173fe3f15ef\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") "
Sep 16 18:53:19.442751 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.442737    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-trusted-ca-bundle\") pod \"17943503-a564-4ab8-b255-1173fe3f15ef\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") "
Sep 16 18:53:19.442950 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.442874    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17943503-a564-4ab8-b255-1173fe3f15ef-console-serving-cert\") pod \"17943503-a564-4ab8-b255-1173fe3f15ef\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") "
Sep 16 18:53:19.443006 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.442954    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17943503-a564-4ab8-b255-1173fe3f15ef-console-oauth-config\") pod \"17943503-a564-4ab8-b255-1173fe3f15ef\" (UID: \"17943503-a564-4ab8-b255-1173fe3f15ef\") "
Sep 16 18:53:19.443057 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.443025    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-console-config" (OuterVolumeSpecName: "console-config") pod "17943503-a564-4ab8-b255-1173fe3f15ef" (UID: "17943503-a564-4ab8-b255-1173fe3f15ef"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:53:19.443111 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.443084    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-service-ca" (OuterVolumeSpecName: "service-ca") pod "17943503-a564-4ab8-b255-1173fe3f15ef" (UID: "17943503-a564-4ab8-b255-1173fe3f15ef"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:53:19.443111 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.443089    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "17943503-a564-4ab8-b255-1173fe3f15ef" (UID: "17943503-a564-4ab8-b255-1173fe3f15ef"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:53:19.443242 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.443142    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "17943503-a564-4ab8-b255-1173fe3f15ef" (UID: "17943503-a564-4ab8-b255-1173fe3f15ef"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:53:19.443369 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.443351    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-console-config\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:53:19.443428 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.443374    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-service-ca\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:53:19.443428 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.443388    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-oauth-serving-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:53:19.443428 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.443403    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17943503-a564-4ab8-b255-1173fe3f15ef-trusted-ca-bundle\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:53:19.444923 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.444899    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17943503-a564-4ab8-b255-1173fe3f15ef-kube-api-access-qh44z" (OuterVolumeSpecName: "kube-api-access-qh44z") pod "17943503-a564-4ab8-b255-1173fe3f15ef" (UID: "17943503-a564-4ab8-b255-1173fe3f15ef"). InnerVolumeSpecName "kube-api-access-qh44z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:53:19.444998 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.444981    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17943503-a564-4ab8-b255-1173fe3f15ef-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "17943503-a564-4ab8-b255-1173fe3f15ef" (UID: "17943503-a564-4ab8-b255-1173fe3f15ef"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:53:19.445037 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.444996    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17943503-a564-4ab8-b255-1173fe3f15ef-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "17943503-a564-4ab8-b255-1173fe3f15ef" (UID: "17943503-a564-4ab8-b255-1173fe3f15ef"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:53:19.544659 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.544570    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh44z\" (UniqueName: \"kubernetes.io/projected/17943503-a564-4ab8-b255-1173fe3f15ef-kube-api-access-qh44z\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:53:19.544659 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.544596    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/17943503-a564-4ab8-b255-1173fe3f15ef-console-serving-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:53:19.544659 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.544605    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/17943503-a564-4ab8-b255-1173fe3f15ef-console-oauth-config\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:53:19.726526 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.726494    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5747b7df4-z87km"]
Sep 16 18:53:19.732777 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:19.732749    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5747b7df4-z87km"]
Sep 16 18:53:20.709295 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:20.709256    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17943503-a564-4ab8-b255-1173fe3f15ef" path="/var/lib/kubelet/pods/17943503-a564-4ab8-b255-1173fe3f15ef/volumes"
Sep 16 18:53:27.394841 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:27.394805    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"
Sep 16 18:53:43.840567 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:43.840534    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"]
Sep 16 18:53:43.841113 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:43.841047    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17943503-a564-4ab8-b255-1173fe3f15ef" containerName="console"
Sep 16 18:53:43.841113 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:43.841064    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="17943503-a564-4ab8-b255-1173fe3f15ef" containerName="console"
Sep 16 18:53:43.841236 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:43.841206    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="17943503-a564-4ab8-b255-1173fe3f15ef" containerName="console"
Sep 16 18:53:43.845528 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:43.845500    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:43.847762 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:43.847735    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wp7w9\""
Sep 16 18:53:43.855387 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:43.855361    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"]
Sep 16 18:53:43.964164 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:43.964127    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85290364-ea9e-4f0f-ae40-fb741b4fbc68-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:43.964376 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:43.964198    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85290364-ea9e-4f0f-ae40-fb741b4fbc68-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:43.964376 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:43.964267    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85290364-ea9e-4f0f-ae40-fb741b4fbc68-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:43.964376 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:43.964322    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85290364-ea9e-4f0f-ae40-fb741b4fbc68-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:43.964535 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:43.964395    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85290364-ea9e-4f0f-ae40-fb741b4fbc68-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:43.964535 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:43.964420    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85290364-ea9e-4f0f-ae40-fb741b4fbc68-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:43.964535 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:43.964460    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85290364-ea9e-4f0f-ae40-fb741b4fbc68-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:43.964699 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:43.964538    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85290364-ea9e-4f0f-ae40-fb741b4fbc68-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:43.964699 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:43.964626    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt4xr\" (UniqueName: \"kubernetes.io/projected/85290364-ea9e-4f0f-ae40-fb741b4fbc68-kube-api-access-nt4xr\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.066089 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.066048    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85290364-ea9e-4f0f-ae40-fb741b4fbc68-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.066089 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.066093    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85290364-ea9e-4f0f-ae40-fb741b4fbc68-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.066350 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.066137    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt4xr\" (UniqueName: \"kubernetes.io/projected/85290364-ea9e-4f0f-ae40-fb741b4fbc68-kube-api-access-nt4xr\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.066350 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.066208    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85290364-ea9e-4f0f-ae40-fb741b4fbc68-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.066350 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.066244    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85290364-ea9e-4f0f-ae40-fb741b4fbc68-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.066350 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.066272    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85290364-ea9e-4f0f-ae40-fb741b4fbc68-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.066350 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.066304    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85290364-ea9e-4f0f-ae40-fb741b4fbc68-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.066592 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.066344    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85290364-ea9e-4f0f-ae40-fb741b4fbc68-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.066592 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.066374    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85290364-ea9e-4f0f-ae40-fb741b4fbc68-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.066702 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.066670    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85290364-ea9e-4f0f-ae40-fb741b4fbc68-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.066810 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.066790    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85290364-ea9e-4f0f-ae40-fb741b4fbc68-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.066881 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.066856    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85290364-ea9e-4f0f-ae40-fb741b4fbc68-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.066939 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.066887    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85290364-ea9e-4f0f-ae40-fb741b4fbc68-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.067077 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.067050    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85290364-ea9e-4f0f-ae40-fb741b4fbc68-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.068468 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.068449    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85290364-ea9e-4f0f-ae40-fb741b4fbc68-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.068948 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.068933    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85290364-ea9e-4f0f-ae40-fb741b4fbc68-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.074560 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.074539    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85290364-ea9e-4f0f-ae40-fb741b4fbc68-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.074773 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.074751    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt4xr\" (UniqueName: \"kubernetes.io/projected/85290364-ea9e-4f0f-ae40-fb741b4fbc68-kube-api-access-nt4xr\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rcxf8\" (UID: \"85290364-ea9e-4f0f-ae40-fb741b4fbc68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.157695 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.157660    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:44.439202 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.439148    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"]
Sep 16 18:53:44.440557 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:53:44.440520    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85290364_ea9e_4f0f_ae40_fb741b4fbc68.slice/crio-7a4bb47bfa5ac686fa6ede403d1484f4349437d7b74ef66c1ebfa6e4a900b1f9 WatchSource:0}: Error finding container 7a4bb47bfa5ac686fa6ede403d1484f4349437d7b74ef66c1ebfa6e4a900b1f9: Status 404 returned error can't find the container with id 7a4bb47bfa5ac686fa6ede403d1484f4349437d7b74ef66c1ebfa6e4a900b1f9
Sep 16 18:53:44.443196 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.443135    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:53:44.443277 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.443240    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:53:44.443327 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.443285    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:53:44.544012 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.543970    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8" event={"ID":"85290364-ea9e-4f0f-ae40-fb741b4fbc68","Type":"ContainerStarted","Data":"d31700c0aeb905e6bf2722d89513046898f9f9f35d22c1eb507d5b4835c3f028"}
Sep 16 18:53:44.544215 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.544040    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8" event={"ID":"85290364-ea9e-4f0f-ae40-fb741b4fbc68","Type":"ContainerStarted","Data":"7a4bb47bfa5ac686fa6ede403d1484f4349437d7b74ef66c1ebfa6e4a900b1f9"}
Sep 16 18:53:44.566217 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:44.566127    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8" podStartSLOduration=1.566106237 podStartE2EDuration="1.566106237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:53:44.565954068 +0000 UTC m=+852.353518224" watchObservedRunningTime="2026-09-16 18:53:44.566106237 +0000 UTC m=+852.353670394"
Sep 16 18:53:45.158060 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:45.158020    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:45.162923 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:45.162897    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:45.549494 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:45.549406    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:53:45.550350 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:53:45.550326    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:54:02.301508 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:02.301468    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sz4pq"]
Sep 16 18:54:02.305754 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:02.305736    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4pq"
Sep 16 18:54:02.309348 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:02.309327    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 16 18:54:02.309479 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:02.309411    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-m7ls9\""
Sep 16 18:54:02.309561 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:02.309545    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 16 18:54:02.309932 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:02.309911    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:54:02.318639 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:02.318615    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sz4pq"]
Sep 16 18:54:02.423332 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:02.423294    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d3908004-d532-480f-a0f4-331659a9a00a-data\") pod \"postgres-69fb4594cf-sz4pq\" (UID: \"d3908004-d532-480f-a0f4-331659a9a00a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4pq"
Sep 16 18:54:02.423524 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:02.423358    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj4kc\" (UniqueName: \"kubernetes.io/projected/d3908004-d532-480f-a0f4-331659a9a00a-kube-api-access-hj4kc\") pod \"postgres-69fb4594cf-sz4pq\" (UID: \"d3908004-d532-480f-a0f4-331659a9a00a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4pq"
Sep 16 18:54:02.524184 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:02.524145    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d3908004-d532-480f-a0f4-331659a9a00a-data\") pod \"postgres-69fb4594cf-sz4pq\" (UID: \"d3908004-d532-480f-a0f4-331659a9a00a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4pq"
Sep 16 18:54:02.524351 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:02.524242    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj4kc\" (UniqueName: \"kubernetes.io/projected/d3908004-d532-480f-a0f4-331659a9a00a-kube-api-access-hj4kc\") pod \"postgres-69fb4594cf-sz4pq\" (UID: \"d3908004-d532-480f-a0f4-331659a9a00a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4pq"
Sep 16 18:54:02.524530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:02.524508    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d3908004-d532-480f-a0f4-331659a9a00a-data\") pod \"postgres-69fb4594cf-sz4pq\" (UID: \"d3908004-d532-480f-a0f4-331659a9a00a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4pq"
Sep 16 18:54:02.532717 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:02.532688    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj4kc\" (UniqueName: \"kubernetes.io/projected/d3908004-d532-480f-a0f4-331659a9a00a-kube-api-access-hj4kc\") pod \"postgres-69fb4594cf-sz4pq\" (UID: \"d3908004-d532-480f-a0f4-331659a9a00a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4pq"
Sep 16 18:54:02.616922 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:02.616889    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4pq"
Sep 16 18:54:02.907228 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:02.907204    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sz4pq"]
Sep 16 18:54:02.908948 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:54:02.908917    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3908004_d532_480f_a0f4_331659a9a00a.slice/crio-f3bab436b7f75f9cc8e40638c62bd61330eb0624acfeb377e060d1fff423d8db WatchSource:0}: Error finding container f3bab436b7f75f9cc8e40638c62bd61330eb0624acfeb377e060d1fff423d8db: Status 404 returned error can't find the container with id f3bab436b7f75f9cc8e40638c62bd61330eb0624acfeb377e060d1fff423d8db
Sep 16 18:54:03.653840 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:03.650757    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4pq" event={"ID":"d3908004-d532-480f-a0f4-331659a9a00a","Type":"ContainerStarted","Data":"f3bab436b7f75f9cc8e40638c62bd61330eb0624acfeb377e060d1fff423d8db"}
Sep 16 18:54:08.682927 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:08.682892    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4pq" event={"ID":"d3908004-d532-480f-a0f4-331659a9a00a","Type":"ContainerStarted","Data":"120fd661da5ee8e49ad38e12aa773887096a488e63a0dbb44919d7fdd51ed42f"}
Sep 16 18:54:08.683376 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:08.682991    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4pq"
Sep 16 18:54:08.719736 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:08.719686    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4pq" podStartSLOduration=1.691916816 podStartE2EDuration="6.719670488s" totalImagesPullingTime="5.027753672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:54:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:54:08.717733115 +0000 UTC m=+876.505297297" watchObservedRunningTime="2026-09-16 18:54:08.719670488 +0000 UTC m=+876.507234643"
Sep 16 18:54:14.718377 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:14.718343    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4pq"
Sep 16 18:54:32.685596 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:32.685561    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 18:54:32.686072 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:32.685830    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 18:54:32.691884 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:32.691859    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 18:54:32.692068 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:54:32.692048    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 18:55:01.046468 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:01.046429    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hctdl"]
Sep 16 18:55:01.049931 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:01.049912    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hctdl"
Sep 16 18:55:01.052164 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:01.052145    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-5kw74\""
Sep 16 18:55:01.052366 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:01.052353    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 16 18:55:01.062698 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:01.062671    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hctdl"]
Sep 16 18:55:01.144916 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:01.144872    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9af41fc-a845-4391-ac4c-7e1ee182f84f-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hctdl\" (UID: \"e9af41fc-a845-4391-ac4c-7e1ee182f84f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hctdl"
Sep 16 18:55:01.145099 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:01.144929    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk4hk\" (UniqueName: \"kubernetes.io/projected/e9af41fc-a845-4391-ac4c-7e1ee182f84f-kube-api-access-tk4hk\") pod \"payload-pre-processing-59fbc47bf8-hctdl\" (UID: \"e9af41fc-a845-4391-ac4c-7e1ee182f84f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hctdl"
Sep 16 18:55:01.245726 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:01.245692    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9af41fc-a845-4391-ac4c-7e1ee182f84f-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hctdl\" (UID: \"e9af41fc-a845-4391-ac4c-7e1ee182f84f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hctdl"
Sep 16 18:55:01.245726 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:01.245729    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk4hk\" (UniqueName: \"kubernetes.io/projected/e9af41fc-a845-4391-ac4c-7e1ee182f84f-kube-api-access-tk4hk\") pod \"payload-pre-processing-59fbc47bf8-hctdl\" (UID: \"e9af41fc-a845-4391-ac4c-7e1ee182f84f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hctdl"
Sep 16 18:55:01.246552 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:01.246524    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9af41fc-a845-4391-ac4c-7e1ee182f84f-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-hctdl\" (UID: \"e9af41fc-a845-4391-ac4c-7e1ee182f84f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hctdl"
Sep 16 18:55:01.257347 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:01.257315    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk4hk\" (UniqueName: \"kubernetes.io/projected/e9af41fc-a845-4391-ac4c-7e1ee182f84f-kube-api-access-tk4hk\") pod \"payload-pre-processing-59fbc47bf8-hctdl\" (UID: \"e9af41fc-a845-4391-ac4c-7e1ee182f84f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hctdl"
Sep 16 18:55:01.361808 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:01.361773    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hctdl"
Sep 16 18:55:01.639718 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:01.638196    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-hctdl"]
Sep 16 18:55:01.649359 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:55:01.649303    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9af41fc_a845_4391_ac4c_7e1ee182f84f.slice/crio-7b003ba2c29ed87543ffb980d4755e881854e7425959cd765371a860c91772a8 WatchSource:0}: Error finding container 7b003ba2c29ed87543ffb980d4755e881854e7425959cd765371a860c91772a8: Status 404 returned error can't find the container with id 7b003ba2c29ed87543ffb980d4755e881854e7425959cd765371a860c91772a8
Sep 16 18:55:01.651061 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:01.651040    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:55:01.985648 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:01.985559    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hctdl" event={"ID":"e9af41fc-a845-4391-ac4c-7e1ee182f84f","Type":"ContainerStarted","Data":"7b003ba2c29ed87543ffb980d4755e881854e7425959cd765371a860c91772a8"}
Sep 16 18:55:03.998570 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:03.998534    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hctdl" event={"ID":"e9af41fc-a845-4391-ac4c-7e1ee182f84f","Type":"ContainerStarted","Data":"86343900b314a1c3bcf04049c7c7474231655ee9449c57ab887bcf4f7f733081"}
Sep 16 18:55:03.999013 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:03.998661    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hctdl"
Sep 16 18:55:04.028293 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:04.028240    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hctdl" podStartSLOduration=1.3854073709999999 podStartE2EDuration="3.02822297s" totalImagesPullingTime="1.642815599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:55:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:55:04.025707739 +0000 UTC m=+931.813271912" watchObservedRunningTime="2026-09-16 18:55:04.02822297 +0000 UTC m=+931.815787185"
Sep 16 18:55:11.040149 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:11.040115    2578 generic.go:356] "Generic (PLEG): container finished" podID="85290364-ea9e-4f0f-ae40-fb741b4fbc68" containerID="d31700c0aeb905e6bf2722d89513046898f9f9f35d22c1eb507d5b4835c3f028" exitCode=0
Sep 16 18:55:11.040608 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:11.040206    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8" event={"ID":"85290364-ea9e-4f0f-ae40-fb741b4fbc68","Type":"ContainerDied","Data":"d31700c0aeb905e6bf2722d89513046898f9f9f35d22c1eb507d5b4835c3f028"}
Sep 16 18:55:11.040654 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:11.040607    2578 scope.go:117] "RemoveContainer" containerID="d31700c0aeb905e6bf2722d89513046898f9f9f35d22c1eb507d5b4835c3f028"
Sep 16 18:55:11.041632 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:11.041600    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:55:11.041734 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:11.041689    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:55:11.041801 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:11.041738    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:55:12.051733 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:12.051689    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8" event={"ID":"85290364-ea9e-4f0f-ae40-fb741b4fbc68","Type":"ContainerStarted","Data":"68f3f4d54dd2b02102f42020d67d37a2dd4605b0a9c8ff6a42664476a13b129c"}
Sep 16 18:55:12.158698 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:12.158617    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:55:12.159952 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:12.159579    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-rcxf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 16 18:55:12.159952 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:12.159648    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8" podUID="85290364-ea9e-4f0f-ae40-fb741b4fbc68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 16 18:55:13.158575 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:13.158526    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-rcxf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 16 18:55:13.159077 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:13.158609    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8" podUID="85290364-ea9e-4f0f-ae40-fb741b4fbc68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 16 18:55:14.159222 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:14.159158    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-rcxf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 16 18:55:14.159843 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:14.159810    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8" podUID="85290364-ea9e-4f0f-ae40-fb741b4fbc68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 16 18:55:15.006759 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:15.006729    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-hctdl"
Sep 16 18:55:15.161685 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:15.161636    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-rcxf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 16 18:55:15.162199 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:15.161720    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8" podUID="85290364-ea9e-4f0f-ae40-fb741b4fbc68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 16 18:55:15.550401 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:15.550365    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:55:16.159213 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:16.159157    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-rcxf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 16 18:55:16.159517 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:16.159485    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8" podUID="85290364-ea9e-4f0f-ae40-fb741b4fbc68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 16 18:55:17.197675 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:17.197628    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:55:17.198760 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:17.198733    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rcxf8"
Sep 16 18:55:33.466600 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.466570    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"]
Sep 16 18:55:33.470044 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.470023    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.472671 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.472650    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lhtq7\""
Sep 16 18:55:33.472942 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.472928    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 16 18:55:33.472999 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.472937    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 16 18:55:33.473267 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.473254    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 16 18:55:33.492701 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.492674    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"]
Sep 16 18:55:33.534445 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.534409    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2045438-5d90-4175-8005-91331a08ac76-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.534641 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.534496    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2045438-5d90-4175-8005-91331a08ac76-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.534641 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.534521    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b6g2\" (UniqueName: \"kubernetes.io/projected/b2045438-5d90-4175-8005-91331a08ac76-kube-api-access-9b6g2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.534641 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.534546    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2045438-5d90-4175-8005-91331a08ac76-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.534641 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.534575    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2045438-5d90-4175-8005-91331a08ac76-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.534641 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.534614    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2045438-5d90-4175-8005-91331a08ac76-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.635952 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.635913    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2045438-5d90-4175-8005-91331a08ac76-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.636155 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.635960    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2045438-5d90-4175-8005-91331a08ac76-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.636155 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.636045    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2045438-5d90-4175-8005-91331a08ac76-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.636155 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.636069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b6g2\" (UniqueName: \"kubernetes.io/projected/b2045438-5d90-4175-8005-91331a08ac76-kube-api-access-9b6g2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.636155 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.636109    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2045438-5d90-4175-8005-91331a08ac76-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.636155 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.636146    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2045438-5d90-4175-8005-91331a08ac76-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.636508 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.636476    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2045438-5d90-4175-8005-91331a08ac76-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.636581 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.636533    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2045438-5d90-4175-8005-91331a08ac76-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.636621 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.636580    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2045438-5d90-4175-8005-91331a08ac76-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.638346 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.638316    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2045438-5d90-4175-8005-91331a08ac76-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.638644 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.638625    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2045438-5d90-4175-8005-91331a08ac76-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.645996 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.645968    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b6g2\" (UniqueName: \"kubernetes.io/projected/b2045438-5d90-4175-8005-91331a08ac76-kube-api-access-9b6g2\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqjl4\" (UID: \"b2045438-5d90-4175-8005-91331a08ac76\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:33.780872 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:33.780778    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:34.071824 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:34.071755    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"]
Sep 16 18:55:34.074367 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:55:34.074339    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2045438_5d90_4175_8005_91331a08ac76.slice/crio-a0d7c7377b1c66188cefda3c2136a2c4c12b62b3017c2aef1f7bf5673f3b80d2 WatchSource:0}: Error finding container a0d7c7377b1c66188cefda3c2136a2c4c12b62b3017c2aef1f7bf5673f3b80d2: Status 404 returned error can't find the container with id a0d7c7377b1c66188cefda3c2136a2c4c12b62b3017c2aef1f7bf5673f3b80d2
Sep 16 18:55:34.187326 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:34.187290    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4" event={"ID":"b2045438-5d90-4175-8005-91331a08ac76","Type":"ContainerStarted","Data":"a0d7c7377b1c66188cefda3c2136a2c4c12b62b3017c2aef1f7bf5673f3b80d2"}
Sep 16 18:55:35.197639 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:35.197604    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4" event={"ID":"b2045438-5d90-4175-8005-91331a08ac76","Type":"ContainerStarted","Data":"2c5a7c97ed68dde76c426593d770622adeff34eaaff8228dd42a4df6166507af"}
Sep 16 18:55:35.229725 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:35.229670    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4" podStartSLOduration=1.2268821380000001 podStartE2EDuration="2.229653494s" totalImagesPullingTime="1.002771356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:55:35.225557699 +0000 UTC m=+963.013121875" watchObservedRunningTime="2026-09-16 18:55:35.229653494 +0000 UTC m=+963.017217650"
Sep 16 18:55:43.780969 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:43.780925    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:43.781550 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:43.780993    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:43.793726 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:43.793694    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:55:44.269570 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:55:44.269535    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqjl4"
Sep 16 18:56:15.591460 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.591424    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"]
Sep 16 18:56:15.595430 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.595409    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.599379 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.599350    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 16 18:56:15.620478 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.620439    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"]
Sep 16 18:56:15.628020 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.627994    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bdcf741-3ca7-463e-ace8-b261652b59c9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.628186 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.628025    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bdcf741-3ca7-463e-ace8-b261652b59c9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.628186 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.628048    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5drq8\" (UniqueName: \"kubernetes.io/projected/7bdcf741-3ca7-463e-ace8-b261652b59c9-kube-api-access-5drq8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.628278 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.628181    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bdcf741-3ca7-463e-ace8-b261652b59c9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.628323 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.628292    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bdcf741-3ca7-463e-ace8-b261652b59c9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.628368 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.628323    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bdcf741-3ca7-463e-ace8-b261652b59c9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.729118 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.729079    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bdcf741-3ca7-463e-ace8-b261652b59c9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.729329 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.729148    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bdcf741-3ca7-463e-ace8-b261652b59c9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.729329 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.729167    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bdcf741-3ca7-463e-ace8-b261652b59c9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.729329 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.729216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bdcf741-3ca7-463e-ace8-b261652b59c9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.729329 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.729232    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bdcf741-3ca7-463e-ace8-b261652b59c9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.729329 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.729247    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5drq8\" (UniqueName: \"kubernetes.io/projected/7bdcf741-3ca7-463e-ace8-b261652b59c9-kube-api-access-5drq8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.729643 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.729614    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bdcf741-3ca7-463e-ace8-b261652b59c9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.729704 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.729677    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bdcf741-3ca7-463e-ace8-b261652b59c9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.729823 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.729803    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bdcf741-3ca7-463e-ace8-b261652b59c9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.731738 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.731709    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bdcf741-3ca7-463e-ace8-b261652b59c9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.731962 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.731937    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bdcf741-3ca7-463e-ace8-b261652b59c9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.745833 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.745743    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5drq8\" (UniqueName: \"kubernetes.io/projected/7bdcf741-3ca7-463e-ace8-b261652b59c9-kube-api-access-5drq8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-t7jpp\" (UID: \"7bdcf741-3ca7-463e-ace8-b261652b59c9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:15.906479 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:15.906440    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:16.186818 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:16.186791    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"]
Sep 16 18:56:16.188185 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:56:16.188140    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bdcf741_3ca7_463e_ace8_b261652b59c9.slice/crio-f93f52f523ac50e707b32036b095aaa0498118d4f3f6d9d1307593b5e0096c40 WatchSource:0}: Error finding container f93f52f523ac50e707b32036b095aaa0498118d4f3f6d9d1307593b5e0096c40: Status 404 returned error can't find the container with id f93f52f523ac50e707b32036b095aaa0498118d4f3f6d9d1307593b5e0096c40
Sep 16 18:56:16.447270 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:16.447187    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp" event={"ID":"7bdcf741-3ca7-463e-ace8-b261652b59c9","Type":"ContainerStarted","Data":"b2a24db5758d2c7cd873a7dff95a0ee80a7e0b57abf6e5c0cc6d1bc3aa87e1c9"}
Sep 16 18:56:16.447270 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:16.447240    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp" event={"ID":"7bdcf741-3ca7-463e-ace8-b261652b59c9","Type":"ContainerStarted","Data":"f93f52f523ac50e707b32036b095aaa0498118d4f3f6d9d1307593b5e0096c40"}
Sep 16 18:56:16.469292 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:16.469235    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp" podStartSLOduration=1.286546563 podStartE2EDuration="1.469215322s" totalImagesPullingTime="182.668759ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:56:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:56:16.466937208 +0000 UTC m=+1004.254501385" watchObservedRunningTime="2026-09-16 18:56:16.469215322 +0000 UTC m=+1004.256779478"
Sep 16 18:56:25.906791 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:25.906750    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:25.907418 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:25.906826    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:25.919591 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:25.919564    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:56:26.522602 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:56:26.522575    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-t7jpp"
Sep 16 18:58:12.520442 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.520401    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"]
Sep 16 18:58:12.523977 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.523951    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.526893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.526869    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-c090c8-openshift-default-dockercfg-njnq9\""
Sep 16 18:58:12.536852 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.536827    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"]
Sep 16 18:58:12.619504 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.619472    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"]
Sep 16 18:58:12.623095 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.623072    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.627223 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.627201    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-ec32e7-openshift-default-dockercfg-bcc6w\""
Sep 16 18:58:12.642337 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.642313    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"]
Sep 16 18:58:12.712532 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.712497    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"]
Sep 16 18:58:12.715889 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.715871    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.719040 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.719018    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-b1857c-openshift-default-dockercfg-g87cz\""
Sep 16 18:58:12.721284 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.721253    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-envoy\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.721530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.721353    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-token\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.721530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.721438    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istiod-ca-cert\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.721530 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.721498    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-podinfo\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.721753 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.721555    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-workload-certs\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.721753 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.721629    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-workload-socket\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.721753 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.721677    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-credential-socket\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.721753 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.721707    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxvpv\" (UniqueName: \"kubernetes.io/projected/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-kube-api-access-jxvpv\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.721923 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.721756    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-data\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.737817 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.737785    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"]
Sep 16 18:58:12.823599 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.823511    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-workload-certs\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.823599 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.823552    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-envoy\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.823599 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.823579    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-credential-socket\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.823882 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.823612    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-envoy\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.823882 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.823628    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-workload-socket\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.823882 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.823645    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-data\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.823882 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.823724    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-workload-socket\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.823882 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.823777    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6pl5\" (UniqueName: \"kubernetes.io/projected/50e70c91-ac88-42eb-8d60-76e5e3869277-kube-api-access-m6pl5\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.823882 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.823797    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxvpv\" (UniqueName: \"kubernetes.io/projected/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-kube-api-access-jxvpv\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.823882 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.823816    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-data\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.823882 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.823838    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-credential-socket\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.823882 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.823864    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-token\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.824349 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.823893    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-workload-certs\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.824349 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.823904    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50e70c91-ac88-42eb-8d60-76e5e3869277-istiod-ca-cert\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.824349 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.823967    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz2xl\" (UniqueName: \"kubernetes.io/projected/c292ec82-49cf-4c8d-9072-6eff2febbe63-kube-api-access-dz2xl\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.824349 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.823974    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-data\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.824349 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.824003    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-token\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.824349 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.824026    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-workload-socket\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.824349 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.824030    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istiod-ca-cert\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.824349 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.824067    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-data\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.824349 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.824104    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-podinfo\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.824349 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.824143    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-credential-socket\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.824349 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.824163    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-workload-socket\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.824349 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.824203    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-envoy\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.824349 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.824231    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c292ec82-49cf-4c8d-9072-6eff2febbe63-istiod-ca-cert\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.824349 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.824254    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-workload-certs\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.825114 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.824290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-podinfo\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.825114 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.824341    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-podinfo\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.825114 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.824400    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-token\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.825114 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.824427    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-workload-certs\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.825114 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.824440    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-credential-socket\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.825114 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.824661    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istiod-ca-cert\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.826521 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.826471    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-envoy\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.826759 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.826741    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-podinfo\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.833256 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.833231    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-token\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.834918 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.834897    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxvpv\" (UniqueName: \"kubernetes.io/projected/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-kube-api-access-jxvpv\") pod \"e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") " pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.836709 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.836686    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:12.927540 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.925979    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-data\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.927540 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926071    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-workload-socket\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.927540 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926107    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c292ec82-49cf-4c8d-9072-6eff2febbe63-istiod-ca-cert\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.927540 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926137    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-workload-certs\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.927540 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926225    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-podinfo\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.927540 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-podinfo\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.927540 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926308    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-token\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.927540 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926335    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-workload-certs\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.927540 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926381    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-envoy\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.927540 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926421    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-credential-socket\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.927540 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926451    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-envoy\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.927540 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926475    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-workload-socket\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.927540 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926508    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6pl5\" (UniqueName: \"kubernetes.io/projected/50e70c91-ac88-42eb-8d60-76e5e3869277-kube-api-access-m6pl5\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.927540 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926539    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-data\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.927540 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926571    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-credential-socket\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.928720 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926598    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-token\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.928720 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926620    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50e70c91-ac88-42eb-8d60-76e5e3869277-istiod-ca-cert\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.928720 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.926655    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz2xl\" (UniqueName: \"kubernetes.io/projected/c292ec82-49cf-4c8d-9072-6eff2febbe63-kube-api-access-dz2xl\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.928720 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.927443    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-data\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.928720 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.927806    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-credential-socket\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.929043 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.929025    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-workload-certs\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.932106 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.929298    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-data\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.932106 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.929555    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-workload-socket\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.932106 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.930404    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c292ec82-49cf-4c8d-9072-6eff2febbe63-istiod-ca-cert\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.932106 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.930665    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-workload-socket\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.932106 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.931139    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-credential-socket\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.932106 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.931431    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-workload-certs\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.932106 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.932053    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50e70c91-ac88-42eb-8d60-76e5e3869277-istiod-ca-cert\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.933467 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.933415    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-envoy\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.941380 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.941320    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-podinfo\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.950524 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.950371    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-podinfo\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.954792 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.954689    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-envoy\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.954976 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.954877    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-token\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.955678 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.955523    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-token\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:12.955678 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.955619    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz2xl\" (UniqueName: \"kubernetes.io/projected/c292ec82-49cf-4c8d-9072-6eff2febbe63-kube-api-access-dz2xl\") pod \"e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") " pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:12.972522 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:12.972465    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6pl5\" (UniqueName: \"kubernetes.io/projected/50e70c91-ac88-42eb-8d60-76e5e3869277-kube-api-access-m6pl5\") pod \"e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") " pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:13.026980 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:13.026939    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:13.148008 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:13.147981    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"]
Sep 16 18:58:13.149864 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:58:13.149831    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda1b180f_d138_4a97_9fe4_1bc43cfeaa4a.slice/crio-33f9f3508a46cab900eef6b6cac63ddbf44303ee50bb1caefb60d3c73f6b003f WatchSource:0}: Error finding container 33f9f3508a46cab900eef6b6cac63ddbf44303ee50bb1caefb60d3c73f6b003f: Status 404 returned error can't find the container with id 33f9f3508a46cab900eef6b6cac63ddbf44303ee50bb1caefb60d3c73f6b003f
Sep 16 18:58:13.152108 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:13.152072    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:13.152235 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:13.152141    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:13.152235 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:13.152198    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:13.235250 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:13.235217    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:13.324901 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:13.324865    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"]
Sep 16 18:58:13.332444 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:58:13.332400    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc292ec82_49cf_4c8d_9072_6eff2febbe63.slice/crio-046b5ec07df9c4dbac322ed2e635b7e2d6119340072828f7ffb3c274d9fcba11 WatchSource:0}: Error finding container 046b5ec07df9c4dbac322ed2e635b7e2d6119340072828f7ffb3c274d9fcba11: Status 404 returned error can't find the container with id 046b5ec07df9c4dbac322ed2e635b7e2d6119340072828f7ffb3c274d9fcba11
Sep 16 18:58:13.335109 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:13.335065    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:13.335245 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:13.335153    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:13.335245 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:13.335219    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:13.542763 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:13.542731    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"]
Sep 16 18:58:13.544114 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:58:13.544088    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50e70c91_ac88_42eb_8d60_76e5e3869277.slice/crio-16be00fe9dc61cb03055154b4e8c07bd1c67ee8429e096ea4306a4c4e9d45267 WatchSource:0}: Error finding container 16be00fe9dc61cb03055154b4e8c07bd1c67ee8429e096ea4306a4c4e9d45267: Status 404 returned error can't find the container with id 16be00fe9dc61cb03055154b4e8c07bd1c67ee8429e096ea4306a4c4e9d45267
Sep 16 18:58:13.546917 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:13.546874    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:13.547022 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:13.546963    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:13.547075 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:13.547038    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:14.126713 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:14.126672    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5" event={"ID":"c292ec82-49cf-4c8d-9072-6eff2febbe63","Type":"ContainerStarted","Data":"4ab56b9bb01ab732241beb530642cc88fb1785b3cc6454143bd6bbd770c7b213"}
Sep 16 18:58:14.126925 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:14.126762    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5" event={"ID":"c292ec82-49cf-4c8d-9072-6eff2febbe63","Type":"ContainerStarted","Data":"046b5ec07df9c4dbac322ed2e635b7e2d6119340072828f7ffb3c274d9fcba11"}
Sep 16 18:58:14.128239 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:14.128214    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t" event={"ID":"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a","Type":"ContainerStarted","Data":"bbc2d1ef731853d906413f8a803096c77fed7a9ef916450b95e3db4830b04569"}
Sep 16 18:58:14.128350 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:14.128265    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t" event={"ID":"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a","Type":"ContainerStarted","Data":"33f9f3508a46cab900eef6b6cac63ddbf44303ee50bb1caefb60d3c73f6b003f"}
Sep 16 18:58:14.129801 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:14.129773    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr" event={"ID":"50e70c91-ac88-42eb-8d60-76e5e3869277","Type":"ContainerStarted","Data":"2fa5e088ae00698d0f12046c880d48237e214b89b1f9987e347dcaa88b2448f7"}
Sep 16 18:58:14.129886 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:14.129822    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr" event={"ID":"50e70c91-ac88-42eb-8d60-76e5e3869277","Type":"ContainerStarted","Data":"16be00fe9dc61cb03055154b4e8c07bd1c67ee8429e096ea4306a4c4e9d45267"}
Sep 16 18:58:14.188599 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:14.188546    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5" podStartSLOduration=2.188530009 podStartE2EDuration="2.188530009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:58:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:58:14.178317349 +0000 UTC m=+1121.965881524" watchObservedRunningTime="2026-09-16 18:58:14.188530009 +0000 UTC m=+1121.976094164"
Sep 16 18:58:14.220663 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:14.220604    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr" podStartSLOduration=2.220587819 podStartE2EDuration="2.220587819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:58:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:58:14.218543527 +0000 UTC m=+1122.006107704" watchObservedRunningTime="2026-09-16 18:58:14.220587819 +0000 UTC m=+1122.008151974"
Sep 16 18:58:14.236072 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:14.236022    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:14.241501 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:14.241467    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:14.257891 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:14.257841    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t" podStartSLOduration=2.257826274 podStartE2EDuration="2.257826274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:58:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:58:14.254798212 +0000 UTC m=+1122.042362387" watchObservedRunningTime="2026-09-16 18:58:14.257826274 +0000 UTC m=+1122.045390429"
Sep 16 18:58:14.837302 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:14.837262    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:14.843253 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:14.843223    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:15.028238 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:15.028167    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:15.033197 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:15.033162    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:15.137082 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:15.136965    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:15.137082 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:15.137025    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:15.137082 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:15.137050    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:15.137743 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:15.137716    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:15.137874 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:15.137784    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:15.137973 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:15.137953    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:22.163042 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.163007    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"]
Sep 16 18:58:22.167751 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.167730    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.171107 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.171079    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-2efa6f-openshift-default-dockercfg-jz5wn\""
Sep 16 18:58:22.225260 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.224340    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-data\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.225260 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.224410    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-token\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.225260 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.224470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba843615-4a02-4976-ae60-8ceabe7afe30-istiod-ca-cert\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.225260 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.224501    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-credential-socket\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.225260 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.224564    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-envoy\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.225260 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.224586    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r7w5\" (UniqueName: \"kubernetes.io/projected/ba843615-4a02-4976-ae60-8ceabe7afe30-kube-api-access-9r7w5\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.225260 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.224659    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-workload-socket\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.225260 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.224724    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-workload-certs\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.225260 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.224781    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-podinfo\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.280576 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.280540    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"]
Sep 16 18:58:22.325826 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.325795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba843615-4a02-4976-ae60-8ceabe7afe30-istiod-ca-cert\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.325826 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.325831    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-credential-socket\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.326067 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.325866    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-envoy\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.326067 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.325903    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r7w5\" (UniqueName: \"kubernetes.io/projected/ba843615-4a02-4976-ae60-8ceabe7afe30-kube-api-access-9r7w5\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.326067 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.325937    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-workload-socket\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.326067 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.325976    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-workload-certs\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.326328 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.326106    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-podinfo\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.326328 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.326263    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-data\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.326443 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.326362    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-credential-socket\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.326443 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.326404    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-workload-certs\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.326443 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.326418    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-token\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.326640 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.326586    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-data\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.326703 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.326677    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba843615-4a02-4976-ae60-8ceabe7afe30-istiod-ca-cert\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.326860 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.326836    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-workload-socket\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.328358 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.328337    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-envoy\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.328682 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.328661    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-podinfo\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.348632 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.348600    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-token\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.349757 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.349728    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r7w5\" (UniqueName: \"kubernetes.io/projected/ba843615-4a02-4976-ae60-8ceabe7afe30-kube-api-access-9r7w5\") pod \"e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") " pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.481339 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.481247    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:22.765188 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.765150    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"]
Sep 16 18:58:22.767375 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:58:22.767342    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba843615_4a02_4976_ae60_8ceabe7afe30.slice/crio-1682b1593725965d53449ffb9495c96a46501faa0fbaa58a907b085a6835059e WatchSource:0}: Error finding container 1682b1593725965d53449ffb9495c96a46501faa0fbaa58a907b085a6835059e: Status 404 returned error can't find the container with id 1682b1593725965d53449ffb9495c96a46501faa0fbaa58a907b085a6835059e
Sep 16 18:58:22.769826 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.769785    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:22.769915 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.769858    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:22.769915 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:22.769895    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:23.198232 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:23.198195    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98" event={"ID":"ba843615-4a02-4976-ae60-8ceabe7afe30","Type":"ContainerStarted","Data":"6d14991454de6ad1ece791a876e46fec3876cca44e77900a37c89471a2544b73"}
Sep 16 18:58:23.198620 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:23.198257    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98" event={"ID":"ba843615-4a02-4976-ae60-8ceabe7afe30","Type":"ContainerStarted","Data":"1682b1593725965d53449ffb9495c96a46501faa0fbaa58a907b085a6835059e"}
Sep 16 18:58:23.227639 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:23.227585    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98" podStartSLOduration=1.227567291 podStartE2EDuration="1.227567291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:58:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:58:23.226265793 +0000 UTC m=+1131.013829967" watchObservedRunningTime="2026-09-16 18:58:23.227567291 +0000 UTC m=+1131.015131445"
Sep 16 18:58:23.482434 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:23.482339    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:23.487216 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:23.487158    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:24.204763 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:24.204733    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:24.205876 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:24.205859    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:24.977743 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:24.977696    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"]
Sep 16 18:58:24.982118 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:24.982031    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:24.984523 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:24.984497    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4300c3c2-openshift-default-dockercfg-dhfzt\""
Sep 16 18:58:24.996098 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:24.996073    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"]
Sep 16 18:58:25.059195 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.058753    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-data\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.059195 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.058808    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-workload-certs\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.059195 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.058865    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2hms\" (UniqueName: \"kubernetes.io/projected/4dd3034e-5724-494f-ac8d-2243d0a0989f-kube-api-access-r2hms\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.059195 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.058910    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-token\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.059195 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.058937    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-credential-socket\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.059195 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.058966    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-podinfo\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.059195 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.059025    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dd3034e-5724-494f-ac8d-2243d0a0989f-istiod-ca-cert\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.059195 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.059052    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-envoy\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.059195 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.059110    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-workload-socket\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.160693 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.160653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-data\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.160951 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.160930    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-workload-certs\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.161099 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.161085    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2hms\" (UniqueName: \"kubernetes.io/projected/4dd3034e-5724-494f-ac8d-2243d0a0989f-kube-api-access-r2hms\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.161257 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.161241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-token\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.161367 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.161353    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-credential-socket\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.161495 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.161472    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-podinfo\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.161655 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.161641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dd3034e-5724-494f-ac8d-2243d0a0989f-istiod-ca-cert\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.161755 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.161742    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-envoy\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.161889 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.161876    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-workload-socket\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.162437 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.162415    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-workload-socket\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.162844 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.162823    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-data\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.163262 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.163241    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-workload-certs\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.164208 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.164187    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-credential-socket\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.165431 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.165406    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dd3034e-5724-494f-ac8d-2243d0a0989f-istiod-ca-cert\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.176595 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.176563    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-envoy\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.177739 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.177679    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-podinfo\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.183740 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.183712    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-token\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.188908 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.188639    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2hms\" (UniqueName: \"kubernetes.io/projected/4dd3034e-5724-494f-ac8d-2243d0a0989f-kube-api-access-r2hms\") pod \"e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") " pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.296942 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.296848    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:25.620840 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.620807    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"]
Sep 16 18:58:25.622132 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:58:25.622100    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd3034e_5724_494f_ac8d_2243d0a0989f.slice/crio-1930463e4e8cdc1d58254e717b0811fd3c66949e668456c0b7bee560d17962cc WatchSource:0}: Error finding container 1930463e4e8cdc1d58254e717b0811fd3c66949e668456c0b7bee560d17962cc: Status 404 returned error can't find the container with id 1930463e4e8cdc1d58254e717b0811fd3c66949e668456c0b7bee560d17962cc
Sep 16 18:58:25.624710 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.624672    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:25.624786 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.624765    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:25.624865 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:25.624847    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:26.218699 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:26.218657    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn" event={"ID":"4dd3034e-5724-494f-ac8d-2243d0a0989f","Type":"ContainerStarted","Data":"41e2a79dc7a5e0b2db109026bf4e50f099b071449cd5c5123e83bc8fb89ac8cf"}
Sep 16 18:58:26.218889 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:26.218758    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn" event={"ID":"4dd3034e-5724-494f-ac8d-2243d0a0989f","Type":"ContainerStarted","Data":"1930463e4e8cdc1d58254e717b0811fd3c66949e668456c0b7bee560d17962cc"}
Sep 16 18:58:26.245086 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:26.245022    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn" podStartSLOduration=2.244997365 podStartE2EDuration="2.244997365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:58:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:58:26.242499676 +0000 UTC m=+1134.030063832" watchObservedRunningTime="2026-09-16 18:58:26.244997365 +0000 UTC m=+1134.032561539"
Sep 16 18:58:26.298023 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:26.297977    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:26.299585 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:26.299550    2578 patch_prober.go:28] interesting pod/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 16 18:58:26.299711 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:26.299620    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn" podUID="4dd3034e-5724-494f-ac8d-2243d0a0989f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 16 18:58:27.315600 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:27.315567    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:28.236565 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:28.236530    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:28.240786 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:28.240753    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:35.309349 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:35.309262    2578 generic.go:356] "Generic (PLEG): container finished" podID="50e70c91-ac88-42eb-8d60-76e5e3869277" containerID="2fa5e088ae00698d0f12046c880d48237e214b89b1f9987e347dcaa88b2448f7" exitCode=0
Sep 16 18:58:35.309747 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:35.309384    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr" event={"ID":"50e70c91-ac88-42eb-8d60-76e5e3869277","Type":"ContainerDied","Data":"2fa5e088ae00698d0f12046c880d48237e214b89b1f9987e347dcaa88b2448f7"}
Sep 16 18:58:35.309922 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:35.309902    2578 scope.go:117] "RemoveContainer" containerID="2fa5e088ae00698d0f12046c880d48237e214b89b1f9987e347dcaa88b2448f7"
Sep 16 18:58:35.311072 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:35.311042    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:35.311165 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:35.311113    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:35.311165 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:35.311146    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:36.326072 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:36.326029    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr" event={"ID":"50e70c91-ac88-42eb-8d60-76e5e3869277","Type":"ContainerStarted","Data":"b4b27564f136806b37aeaf535e1634a0fc47ae812dd5a27aba9ab60ecdfbc85f"}
Sep 16 18:58:36.327854 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:36.327827    2578 generic.go:356] "Generic (PLEG): container finished" podID="c292ec82-49cf-4c8d-9072-6eff2febbe63" containerID="4ab56b9bb01ab732241beb530642cc88fb1785b3cc6454143bd6bbd770c7b213" exitCode=0
Sep 16 18:58:36.327967 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:36.327900    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5" event={"ID":"c292ec82-49cf-4c8d-9072-6eff2febbe63","Type":"ContainerDied","Data":"4ab56b9bb01ab732241beb530642cc88fb1785b3cc6454143bd6bbd770c7b213"}
Sep 16 18:58:36.328319 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:36.328304    2578 scope.go:117] "RemoveContainer" containerID="4ab56b9bb01ab732241beb530642cc88fb1785b3cc6454143bd6bbd770c7b213"
Sep 16 18:58:36.329398 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:36.329370    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:36.329450 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:36.329438    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:36.329495 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:36.329468    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:37.236028 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.235981    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:37.236240 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.236219    2578 patch_prober.go:28] interesting pod/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 16 18:58:37.236303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.236275    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr" podUID="50e70c91-ac88-42eb-8d60-76e5e3869277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 16 18:58:37.356383 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.356347    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"]
Sep 16 18:58:37.363046 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.363018    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.369849 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.369822    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-8a45a9-openshift-default-dockercfg-b8f7x\""
Sep 16 18:58:37.373335 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.373305    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5" event={"ID":"c292ec82-49cf-4c8d-9072-6eff2febbe63","Type":"ContainerStarted","Data":"18eb0365636b2279141ad6369a0d31053f9e1138ab307187142be0d3dcd2b77a"}
Sep 16 18:58:37.402550 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.402509    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"]
Sep 16 18:58:37.411820 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.411781    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6xkx\" (UniqueName: \"kubernetes.io/projected/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-kube-api-access-x6xkx\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.412218 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.412191    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-podinfo\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.412379 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.412362    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istiod-ca-cert\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.412497 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.412483    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-workload-socket\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.412617 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.412603    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-workload-certs\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.412828 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.412811    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-credential-socket\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.412955 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.412940    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-data\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.413088 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.413072    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-envoy\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.413223 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.413206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-token\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.519321 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.517756    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-workload-socket\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.519321 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.517813    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-workload-certs\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.519321 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.517881    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-credential-socket\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.519321 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.517920    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-data\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.519321 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.517963    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-envoy\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.519321 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.517989    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-token\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.519321 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.518089    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6xkx\" (UniqueName: \"kubernetes.io/projected/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-kube-api-access-x6xkx\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.519321 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.518190    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-podinfo\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.519321 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.518216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istiod-ca-cert\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.519321 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.519085    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istiod-ca-cert\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.524196 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.520148    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-workload-socket\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.524196 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.520408    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-workload-certs\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.524196 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.520615    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-credential-socket\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.524196 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.520801    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-data\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.548132 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.548062    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-envoy\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.557042 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.556946    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6xkx\" (UniqueName: \"kubernetes.io/projected/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-kube-api-access-x6xkx\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.570213 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.566804    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-podinfo\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.580822 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.580741    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-token\") pod \"e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") " pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:37.686851 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:37.686516    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:38.028524 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:38.028439    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:38.029141 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:38.028801    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 16 18:58:38.029141 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:38.028862    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5" podUID="c292ec82-49cf-4c8d-9072-6eff2febbe63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 16 18:58:38.161910 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:38.161870    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"]
Sep 16 18:58:38.165525 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:58:38.165481    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e716728_4c89_4ce6_a53d_e9d8cc7a0731.slice/crio-0b84f988da673b39272a970101c5f81a5dade060a9692feb0f365a9ee46f2cb4 WatchSource:0}: Error finding container 0b84f988da673b39272a970101c5f81a5dade060a9692feb0f365a9ee46f2cb4: Status 404 returned error can't find the container with id 0b84f988da673b39272a970101c5f81a5dade060a9692feb0f365a9ee46f2cb4
Sep 16 18:58:38.175195 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:38.172400    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:38.175195 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:38.172504    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:38.175195 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:38.172548    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:38.241648 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:38.241613    2578 patch_prober.go:28] interesting pod/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 16 18:58:38.241921 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:38.241890    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr" podUID="50e70c91-ac88-42eb-8d60-76e5e3869277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 16 18:58:38.389930 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:38.389886    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm" event={"ID":"7e716728-4c89-4ce6-a53d-e9d8cc7a0731","Type":"ContainerStarted","Data":"0b84f988da673b39272a970101c5f81a5dade060a9692feb0f365a9ee46f2cb4"}
Sep 16 18:58:39.027987 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:39.027950    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 16 18:58:39.028263 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:39.028230    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5" podUID="c292ec82-49cf-4c8d-9072-6eff2febbe63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 16 18:58:39.236894 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:39.236847    2578 patch_prober.go:28] interesting pod/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 16 18:58:39.237103 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:39.236934    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr" podUID="50e70c91-ac88-42eb-8d60-76e5e3869277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 16 18:58:39.401532 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:39.401473    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm" event={"ID":"7e716728-4c89-4ce6-a53d-e9d8cc7a0731","Type":"ContainerStarted","Data":"96078b8777467eb5c4d0330d02fc9a2d310ac02431160bf1bff2d6e84985aa35"}
Sep 16 18:58:39.442133 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:39.442065    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm" podStartSLOduration=2.442026036 podStartE2EDuration="2.442026036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:58:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:58:39.439691534 +0000 UTC m=+1147.227255715" watchObservedRunningTime="2026-09-16 18:58:39.442026036 +0000 UTC m=+1147.229590192"
Sep 16 18:58:39.690370 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:39.690271    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:39.698573 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:39.698542    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:40.027819 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:40.027787    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 16 18:58:40.027965 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:40.027839    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5" podUID="c292ec82-49cf-4c8d-9072-6eff2febbe63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 16 18:58:40.235828 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:40.235786    2578 patch_prober.go:28] interesting pod/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 16 18:58:40.236022 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:40.235841    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr" podUID="50e70c91-ac88-42eb-8d60-76e5e3869277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 16 18:58:40.410926 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:40.410881    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:40.412929 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:40.412902    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:58:41.028141 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:41.028097    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 16 18:58:41.028361 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:41.028190    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5" podUID="c292ec82-49cf-4c8d-9072-6eff2febbe63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 16 18:58:41.236245 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:41.235793    2578 patch_prober.go:28] interesting pod/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 16 18:58:41.236245 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:41.235858    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr" podUID="50e70c91-ac88-42eb-8d60-76e5e3869277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 16 18:58:42.027535 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:42.027492    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 16 18:58:42.028026 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:42.027562    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5" podUID="c292ec82-49cf-4c8d-9072-6eff2febbe63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 16 18:58:42.236458 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:42.236406    2578 patch_prober.go:28] interesting pod/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 16 18:58:42.236672 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:42.236476    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr" podUID="50e70c91-ac88-42eb-8d60-76e5e3869277" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 16 18:58:43.027873 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:43.027830    2578 patch_prober.go:28] interesting pod/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 16 18:58:43.028398 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:43.027904    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5" podUID="c292ec82-49cf-4c8d-9072-6eff2febbe63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 16 18:58:43.056663 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:43.056622    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl"]
Sep 16 18:58:43.060442 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:43.060418    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl"
Sep 16 18:58:43.063917 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:43.063893    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nmr62\""
Sep 16 18:58:43.066509 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:43.066275    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl"]
Sep 16 18:58:43.164839 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:43.164795    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a00baff0-4bb5-4b94-9d02-bff1f60e5dc2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl\" (UID: \"a00baff0-4bb5-4b94-9d02-bff1f60e5dc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl"
Sep 16 18:58:43.243281 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:43.241678    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:43.243281 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:43.243104    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:43.244287 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:43.244237    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 18:58:43.267198 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:43.266864    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a00baff0-4bb5-4b94-9d02-bff1f60e5dc2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl\" (UID: \"a00baff0-4bb5-4b94-9d02-bff1f60e5dc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl"
Sep 16 18:58:43.267773 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:43.267740    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a00baff0-4bb5-4b94-9d02-bff1f60e5dc2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl\" (UID: \"a00baff0-4bb5-4b94-9d02-bff1f60e5dc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl"
Sep 16 18:58:43.379842 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:43.379805    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl"
Sep 16 18:58:43.706753 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:43.706726    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl"]
Sep 16 18:58:43.707725 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:58:43.707678    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda00baff0_4bb5_4b94_9d02_bff1f60e5dc2.slice/crio-a210368e53114eb877508b09bcf19d9874f570d993dd6ddf6e37e6008ba5018a WatchSource:0}: Error finding container a210368e53114eb877508b09bcf19d9874f570d993dd6ddf6e37e6008ba5018a: Status 404 returned error can't find the container with id a210368e53114eb877508b09bcf19d9874f570d993dd6ddf6e37e6008ba5018a
Sep 16 18:58:44.033129 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:44.033047    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:44.033571 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:44.033541    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:44.034386 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:44.034367    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 18:58:44.445502 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:44.445434    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl" event={"ID":"a00baff0-4bb5-4b94-9d02-bff1f60e5dc2","Type":"ContainerStarted","Data":"a210368e53114eb877508b09bcf19d9874f570d993dd6ddf6e37e6008ba5018a"}
Sep 16 18:58:45.138052 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:45.137968    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 16 18:58:45.138052 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:45.138039    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t" podUID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 16 18:58:45.464656 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:45.464556    2578 generic.go:356] "Generic (PLEG): container finished" podID="ba843615-4a02-4976-ae60-8ceabe7afe30" containerID="6d14991454de6ad1ece791a876e46fec3876cca44e77900a37c89471a2544b73" exitCode=0
Sep 16 18:58:45.464656 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:45.464633    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98" event={"ID":"ba843615-4a02-4976-ae60-8ceabe7afe30","Type":"ContainerDied","Data":"6d14991454de6ad1ece791a876e46fec3876cca44e77900a37c89471a2544b73"}
Sep 16 18:58:45.465141 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:45.465122    2578 scope.go:117] "RemoveContainer" containerID="6d14991454de6ad1ece791a876e46fec3876cca44e77900a37c89471a2544b73"
Sep 16 18:58:45.466590 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:45.466559    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:45.466693 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:45.466650    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:45.466763 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:45.466705    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:45.467266 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:45.467240    2578 generic.go:356] "Generic (PLEG): container finished" podID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" containerID="bbc2d1ef731853d906413f8a803096c77fed7a9ef916450b95e3db4830b04569" exitCode=0
Sep 16 18:58:45.467344 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:45.467317    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t" event={"ID":"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a","Type":"ContainerDied","Data":"bbc2d1ef731853d906413f8a803096c77fed7a9ef916450b95e3db4830b04569"}
Sep 16 18:58:45.467909 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:45.467855    2578 scope.go:117] "RemoveContainer" containerID="bbc2d1ef731853d906413f8a803096c77fed7a9ef916450b95e3db4830b04569"
Sep 16 18:58:45.469062 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:45.469026    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:45.469164 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:45.469104    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:45.469164 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:45.469153    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:46.484938 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:46.484896    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t" event={"ID":"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a","Type":"ContainerStarted","Data":"0a9c3177f65844b4805b93190facdd9af4f701878a18cf4cf06a28404179bef6"}
Sep 16 18:58:46.504196 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:46.500762    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98" event={"ID":"ba843615-4a02-4976-ae60-8ceabe7afe30","Type":"ContainerStarted","Data":"8d5f4cd6314b3f1faf8c18f3f3e6ecffea1f2486d68eb19ed1af43a19192083d"}
Sep 16 18:58:46.838250 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:46.837764    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:46.838250 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:46.837886    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 16 18:58:46.838250 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:46.837931    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t" podUID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 16 18:58:47.494199 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:47.481717    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:47.494199 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:47.481945    2578 patch_prober.go:28] interesting pod/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 16 18:58:47.494199 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:47.481997    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98" podUID="ba843615-4a02-4976-ae60-8ceabe7afe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 16 18:58:47.845253 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:47.842599    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 16 18:58:47.845253 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:47.842666    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t" podUID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 16 18:58:48.483142 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:48.482664    2578 patch_prober.go:28] interesting pod/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 16 18:58:48.483142 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:48.482731    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98" podUID="ba843615-4a02-4976-ae60-8ceabe7afe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 16 18:58:48.837361 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:48.837315    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 16 18:58:48.837846 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:48.837388    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t" podUID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 16 18:58:49.483611 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:49.483511    2578 patch_prober.go:28] interesting pod/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 16 18:58:49.483878 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:49.483842    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98" podUID="ba843615-4a02-4976-ae60-8ceabe7afe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 16 18:58:49.531103 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:49.531072    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl" event={"ID":"a00baff0-4bb5-4b94-9d02-bff1f60e5dc2","Type":"ContainerStarted","Data":"64297f4f79e753c24d1366743680c275375c2e4566ad68c7a12879e1600a5fec"}
Sep 16 18:58:49.837641 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:49.837604    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 16 18:58:49.838315 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:49.838256    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t" podUID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 16 18:58:50.482731 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:50.482689    2578 patch_prober.go:28] interesting pod/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 16 18:58:50.483016 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:50.482988    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98" podUID="ba843615-4a02-4976-ae60-8ceabe7afe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 16 18:58:50.539899 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:50.539854    2578 generic.go:356] "Generic (PLEG): container finished" podID="a00baff0-4bb5-4b94-9d02-bff1f60e5dc2" containerID="64297f4f79e753c24d1366743680c275375c2e4566ad68c7a12879e1600a5fec" exitCode=0
Sep 16 18:58:50.540103 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:50.540021    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl" event={"ID":"a00baff0-4bb5-4b94-9d02-bff1f60e5dc2","Type":"ContainerDied","Data":"64297f4f79e753c24d1366743680c275375c2e4566ad68c7a12879e1600a5fec"}
Sep 16 18:58:50.837410 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:50.837313    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 16 18:58:50.837689 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:50.837659    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t" podUID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 16 18:58:51.482804 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:51.482755    2578 patch_prober.go:28] interesting pod/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 16 18:58:51.483026 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:51.482841    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98" podUID="ba843615-4a02-4976-ae60-8ceabe7afe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 16 18:58:51.845371 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:51.845283    2578 patch_prober.go:28] interesting pod/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 16 18:58:51.845960 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:51.845928    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t" podUID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 16 18:58:51.924662 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:51.924634    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl"
Sep 16 18:58:52.018756 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:52.018714    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a00baff0-4bb5-4b94-9d02-bff1f60e5dc2-maas-api-service-ca\") pod \"a00baff0-4bb5-4b94-9d02-bff1f60e5dc2\" (UID: \"a00baff0-4bb5-4b94-9d02-bff1f60e5dc2\") "
Sep 16 18:58:52.019218 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:52.019188    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a00baff0-4bb5-4b94-9d02-bff1f60e5dc2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a00baff0-4bb5-4b94-9d02-bff1f60e5dc2" (UID: "a00baff0-4bb5-4b94-9d02-bff1f60e5dc2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:58:52.120917 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:52.120888    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a00baff0-4bb5-4b94-9d02-bff1f60e5dc2-maas-api-service-ca\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:58:52.486407 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:52.486321    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:52.486759 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:52.486736    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:52.487527 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:52.487510    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 18:58:52.558684 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:52.558648    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl"
Sep 16 18:58:52.558864 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:52.558701    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl" event={"ID":"a00baff0-4bb5-4b94-9d02-bff1f60e5dc2","Type":"ContainerDied","Data":"a210368e53114eb877508b09bcf19d9874f570d993dd6ddf6e37e6008ba5018a"}
Sep 16 18:58:52.558864 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:52.558747    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a210368e53114eb877508b09bcf19d9874f570d993dd6ddf6e37e6008ba5018a"
Sep 16 18:58:52.842690 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:52.842605    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:52.843027 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:52.842994    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:52.843910 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:52.843887    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 18:58:53.447668 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:53.447622    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl"]
Sep 16 18:58:53.452814 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:53.452762    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4300c3c2-bdzdl"]
Sep 16 18:58:53.634507 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:53.634466    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"]
Sep 16 18:58:53.634777 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:53.634747    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn" podUID="4dd3034e-5724-494f-ac8d-2243d0a0989f" containerName="istio-proxy" containerID="cri-o://41e2a79dc7a5e0b2db109026bf4e50f099b071449cd5c5123e83bc8fb89ac8cf" gracePeriod=30
Sep 16 18:58:54.652655 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.652612    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"]
Sep 16 18:58:54.653329 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.653308    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a00baff0-4bb5-4b94-9d02-bff1f60e5dc2" containerName="revoke-keys"
Sep 16 18:58:54.653329 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.653330    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a00baff0-4bb5-4b94-9d02-bff1f60e5dc2" containerName="revoke-keys"
Sep 16 18:58:54.653501 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.653441    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a00baff0-4bb5-4b94-9d02-bff1f60e5dc2" containerName="revoke-keys"
Sep 16 18:58:54.656093 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.656075    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.659355 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.659322    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-78ad8baa-gw-openshift-default-dockercfg-2lhsc\""
Sep 16 18:58:54.670454 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.670428    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"]
Sep 16 18:58:54.711786 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.711752    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a00baff0-4bb5-4b94-9d02-bff1f60e5dc2" path="/var/lib/kubelet/pods/a00baff0-4bb5-4b94-9d02-bff1f60e5dc2/volumes"
Sep 16 18:58:54.854488 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.854395    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-workload-certs\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.854488 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.854449    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjjh9\" (UniqueName: \"kubernetes.io/projected/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-kube-api-access-sjjh9\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.854715 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.854521    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istiod-ca-cert\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.854715 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.854548    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-envoy\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.854715 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.854619    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-token\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.854715 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.854651    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-workload-socket\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.854928 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.854738    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-data\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.854928 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.854794    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-credential-socket\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.854928 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.854916    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-podinfo\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.956570 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.956479    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-workload-socket\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.956570 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.956533    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-data\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.956806 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.956581    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-credential-socket\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.956806 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.956614    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-podinfo\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.956806 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.956658    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-workload-certs\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.956806 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.956695    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjjh9\" (UniqueName: \"kubernetes.io/projected/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-kube-api-access-sjjh9\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.956806 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.956736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istiod-ca-cert\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.956806 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.956763    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-envoy\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.957135 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.956825    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-token\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.957303 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.957276    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-workload-socket\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.957585 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.957557    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istiod-ca-cert\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.957704 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.957593    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-data\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.957704 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.957619    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-workload-certs\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.957824 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.957802    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-credential-socket\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.959363 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.959343    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-podinfo\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.959824 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.959804    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-envoy\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.976639 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.976546    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-token\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:54.977536 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:54.977509    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjjh9\" (UniqueName: \"kubernetes.io/projected/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-kube-api-access-sjjh9\") pod \"e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") " pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:55.270987 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.270899    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:55.582487 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.582460    2578 generic.go:356] "Generic (PLEG): container finished" podID="4dd3034e-5724-494f-ac8d-2243d0a0989f" containerID="41e2a79dc7a5e0b2db109026bf4e50f099b071449cd5c5123e83bc8fb89ac8cf" exitCode=0
Sep 16 18:58:55.582662 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.582502    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn" event={"ID":"4dd3034e-5724-494f-ac8d-2243d0a0989f","Type":"ContainerDied","Data":"41e2a79dc7a5e0b2db109026bf4e50f099b071449cd5c5123e83bc8fb89ac8cf"}
Sep 16 18:58:55.582903 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.582880    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"]
Sep 16 18:58:55.583581 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:58:55.583557    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod182d37f7_1a2a_4c6c_a20a_656fd0ee5354.slice/crio-11d8c626726e8675e0440779e4655908de03c8e08160a4940b610da08acf6bde WatchSource:0}: Error finding container 11d8c626726e8675e0440779e4655908de03c8e08160a4940b610da08acf6bde: Status 404 returned error can't find the container with id 11d8c626726e8675e0440779e4655908de03c8e08160a4940b610da08acf6bde
Sep 16 18:58:55.585812 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.585780    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:55.585896 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.585843    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:55.585896 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.585872    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:58:55.685751 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.685725    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:55.868574 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.866776    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-data\") pod \"4dd3034e-5724-494f-ac8d-2243d0a0989f\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") "
Sep 16 18:58:55.868574 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.866834    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-token\") pod \"4dd3034e-5724-494f-ac8d-2243d0a0989f\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") "
Sep 16 18:58:55.868574 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.866870    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-podinfo\") pod \"4dd3034e-5724-494f-ac8d-2243d0a0989f\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") "
Sep 16 18:58:55.868574 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.866911    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-workload-socket\") pod \"4dd3034e-5724-494f-ac8d-2243d0a0989f\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") "
Sep 16 18:58:55.868574 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.866972    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2hms\" (UniqueName: \"kubernetes.io/projected/4dd3034e-5724-494f-ac8d-2243d0a0989f-kube-api-access-r2hms\") pod \"4dd3034e-5724-494f-ac8d-2243d0a0989f\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") "
Sep 16 18:58:55.868574 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.867065    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-envoy\") pod \"4dd3034e-5724-494f-ac8d-2243d0a0989f\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") "
Sep 16 18:58:55.868574 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.867097    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dd3034e-5724-494f-ac8d-2243d0a0989f-istiod-ca-cert\") pod \"4dd3034e-5724-494f-ac8d-2243d0a0989f\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") "
Sep 16 18:58:55.868574 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.867133    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-credential-socket\") pod \"4dd3034e-5724-494f-ac8d-2243d0a0989f\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") "
Sep 16 18:58:55.868574 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.867255    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-workload-certs\") pod \"4dd3034e-5724-494f-ac8d-2243d0a0989f\" (UID: \"4dd3034e-5724-494f-ac8d-2243d0a0989f\") "
Sep 16 18:58:55.868574 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.868012    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4dd3034e-5724-494f-ac8d-2243d0a0989f" (UID: "4dd3034e-5724-494f-ac8d-2243d0a0989f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:58:55.868574 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.868209    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-data" (OuterVolumeSpecName: "istio-data") pod "4dd3034e-5724-494f-ac8d-2243d0a0989f" (UID: "4dd3034e-5724-494f-ac8d-2243d0a0989f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:58:55.872364 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.872114    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4dd3034e-5724-494f-ac8d-2243d0a0989f" (UID: "4dd3034e-5724-494f-ac8d-2243d0a0989f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 18:58:55.872790 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.872580    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4dd3034e-5724-494f-ac8d-2243d0a0989f" (UID: "4dd3034e-5724-494f-ac8d-2243d0a0989f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:58:55.873345 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.873067    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dd3034e-5724-494f-ac8d-2243d0a0989f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4dd3034e-5724-494f-ac8d-2243d0a0989f" (UID: "4dd3034e-5724-494f-ac8d-2243d0a0989f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:58:55.873604 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.873576    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dd3034e-5724-494f-ac8d-2243d0a0989f-kube-api-access-r2hms" (OuterVolumeSpecName: "kube-api-access-r2hms") pod "4dd3034e-5724-494f-ac8d-2243d0a0989f" (UID: "4dd3034e-5724-494f-ac8d-2243d0a0989f"). InnerVolumeSpecName "kube-api-access-r2hms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:58:55.873822 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.873794    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4dd3034e-5724-494f-ac8d-2243d0a0989f" (UID: "4dd3034e-5724-494f-ac8d-2243d0a0989f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:58:55.877057 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.877031    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-token" (OuterVolumeSpecName: "istio-token") pod "4dd3034e-5724-494f-ac8d-2243d0a0989f" (UID: "4dd3034e-5724-494f-ac8d-2243d0a0989f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:58:55.877145 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.877036    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4dd3034e-5724-494f-ac8d-2243d0a0989f" (UID: "4dd3034e-5724-494f-ac8d-2243d0a0989f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:58:55.968842 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.968814    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:58:55.968983 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.968845    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dd3034e-5724-494f-ac8d-2243d0a0989f-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:58:55.968983 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.968859    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:58:55.968983 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.968871    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:58:55.968983 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.968882    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:58:55.968983 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.968894    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:58:55.968983 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.968904    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dd3034e-5724-494f-ac8d-2243d0a0989f-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:58:55.968983 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.968916    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dd3034e-5724-494f-ac8d-2243d0a0989f-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:58:55.968983 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:55.968929    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2hms\" (UniqueName: \"kubernetes.io/projected/4dd3034e-5724-494f-ac8d-2243d0a0989f-kube-api-access-r2hms\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:58:56.590600 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:56.590557    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"
Sep 16 18:58:56.590600 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:56.590568    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn" event={"ID":"4dd3034e-5724-494f-ac8d-2243d0a0989f","Type":"ContainerDied","Data":"1930463e4e8cdc1d58254e717b0811fd3c66949e668456c0b7bee560d17962cc"}
Sep 16 18:58:56.590881 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:56.590657    2578 scope.go:117] "RemoveContainer" containerID="41e2a79dc7a5e0b2db109026bf4e50f099b071449cd5c5123e83bc8fb89ac8cf"
Sep 16 18:58:56.592781 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:56.592757    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk" event={"ID":"182d37f7-1a2a-4c6c-a20a-656fd0ee5354","Type":"ContainerStarted","Data":"932e083655d975af0720b452f0236b95fd59eec437b400fba95b8deff65a021b"}
Sep 16 18:58:56.592896 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:56.592801    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk" event={"ID":"182d37f7-1a2a-4c6c-a20a-656fd0ee5354","Type":"ContainerStarted","Data":"11d8c626726e8675e0440779e4655908de03c8e08160a4940b610da08acf6bde"}
Sep 16 18:58:56.629964 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:56.629915    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk" podStartSLOduration=2.629898875 podStartE2EDuration="2.629898875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:58:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:58:56.627079002 +0000 UTC m=+1164.414643168" watchObservedRunningTime="2026-09-16 18:58:56.629898875 +0000 UTC m=+1164.417463030"
Sep 16 18:58:56.647162 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:56.647121    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"]
Sep 16 18:58:56.650392 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:56.650361    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4300c3c2-openshift-default-b7cdfc64d-nsqcn"]
Sep 16 18:58:56.710039 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:56.710006    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dd3034e-5724-494f-ac8d-2243d0a0989f" path="/var/lib/kubelet/pods/4dd3034e-5724-494f-ac8d-2243d0a0989f/volumes"
Sep 16 18:58:57.271855 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:57.271810    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:57.277294 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:57.277263    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:57.600756 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:57.600669    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:58:57.601778 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:58:57.601756    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:59:06.671237 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:06.669390    2578 generic.go:356] "Generic (PLEG): container finished" podID="7e716728-4c89-4ce6-a53d-e9d8cc7a0731" containerID="96078b8777467eb5c4d0330d02fc9a2d310ac02431160bf1bff2d6e84985aa35" exitCode=0
Sep 16 18:59:06.671237 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:06.669618    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm" event={"ID":"7e716728-4c89-4ce6-a53d-e9d8cc7a0731","Type":"ContainerDied","Data":"96078b8777467eb5c4d0330d02fc9a2d310ac02431160bf1bff2d6e84985aa35"}
Sep 16 18:59:06.671237 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:06.670244    2578 scope.go:117] "RemoveContainer" containerID="96078b8777467eb5c4d0330d02fc9a2d310ac02431160bf1bff2d6e84985aa35"
Sep 16 18:59:06.674202 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:06.672955    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:59:06.674202 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:06.673073    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:59:06.674202 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:06.673119    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:59:07.678327 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:07.678287    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm" event={"ID":"7e716728-4c89-4ce6-a53d-e9d8cc7a0731","Type":"ContainerStarted","Data":"c67d850975468a65e84e38176a5086a98c9e8597b123f68a06b935792aae81b1"}
Sep 16 18:59:07.687928 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:07.687902    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:59:07.689222 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:07.689198    2578 patch_prober.go:28] interesting pod/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 16 18:59:07.689352 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:07.689248    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm" podUID="7e716728-4c89-4ce6-a53d-e9d8cc7a0731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 16 18:59:08.687977 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:08.687943    2578 patch_prober.go:28] interesting pod/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 16 18:59:08.688389 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:08.687995    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm" podUID="7e716728-4c89-4ce6-a53d-e9d8cc7a0731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 16 18:59:09.687804 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:09.687770    2578 patch_prober.go:28] interesting pod/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 16 18:59:09.687993 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:09.687851    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm" podUID="7e716728-4c89-4ce6-a53d-e9d8cc7a0731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 16 18:59:10.410812 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:10.410776    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:59:10.687995 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:10.687907    2578 patch_prober.go:28] interesting pod/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 16 18:59:10.687995 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:10.687962    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm" podUID="7e716728-4c89-4ce6-a53d-e9d8cc7a0731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 16 18:59:11.687984 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:11.687946    2578 patch_prober.go:28] interesting pod/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 16 18:59:11.688281 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:11.688005    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm" podUID="7e716728-4c89-4ce6-a53d-e9d8cc7a0731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 16 18:59:12.687742 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:12.687705    2578 patch_prober.go:28] interesting pod/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 16 18:59:12.687961 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:12.687769    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm" podUID="7e716728-4c89-4ce6-a53d-e9d8cc7a0731" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 16 18:59:13.691259 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:13.691228    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:59:13.692465 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:13.692443    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 18:59:23.795464 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:23.795421    2578 generic.go:356] "Generic (PLEG): container finished" podID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" containerID="932e083655d975af0720b452f0236b95fd59eec437b400fba95b8deff65a021b" exitCode=0
Sep 16 18:59:23.795895 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:23.795504    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk" event={"ID":"182d37f7-1a2a-4c6c-a20a-656fd0ee5354","Type":"ContainerDied","Data":"932e083655d975af0720b452f0236b95fd59eec437b400fba95b8deff65a021b"}
Sep 16 18:59:23.795989 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:23.795974    2578 scope.go:117] "RemoveContainer" containerID="932e083655d975af0720b452f0236b95fd59eec437b400fba95b8deff65a021b"
Sep 16 18:59:23.797304 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:23.797276    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:59:23.797364 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:23.797345    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:59:23.797401 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:23.797375    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:59:24.807564 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:24.807529    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk" event={"ID":"182d37f7-1a2a-4c6c-a20a-656fd0ee5354","Type":"ContainerStarted","Data":"2e965935314d2b4de02df1be88e737e6bd5b42ebe9fc625d84413c02d752d706"}
Sep 16 18:59:25.271416 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:25.271366    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:59:25.272692 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:25.272666    2578 patch_prober.go:28] interesting pod/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 16 18:59:25.272767 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:25.272734    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk" podUID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 16 18:59:26.272078 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:26.272036    2578 patch_prober.go:28] interesting pod/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 16 18:59:26.272505 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:26.272110    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk" podUID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 16 18:59:27.271827 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:27.271788    2578 patch_prober.go:28] interesting pod/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 16 18:59:27.272004 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:27.271843    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk" podUID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 16 18:59:27.601423 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:27.601321    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:59:28.271895 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:28.271851    2578 patch_prober.go:28] interesting pod/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 16 18:59:28.272111 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:28.271916    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk" podUID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 16 18:59:28.320381 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:28.320339    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j"]
Sep 16 18:59:28.320802 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:28.320789    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dd3034e-5724-494f-ac8d-2243d0a0989f" containerName="istio-proxy"
Sep 16 18:59:28.320846 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:28.320804    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dd3034e-5724-494f-ac8d-2243d0a0989f" containerName="istio-proxy"
Sep 16 18:59:28.320893 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:28.320882    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dd3034e-5724-494f-ac8d-2243d0a0989f" containerName="istio-proxy"
Sep 16 18:59:28.322987 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:28.322971    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j"
Sep 16 18:59:28.326420 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:28.326399    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59c70509-13d1-4993-80f5-2ba78b244112-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j\" (UID: \"59c70509-13d1-4993-80f5-2ba78b244112\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j"
Sep 16 18:59:28.327858 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:28.327838    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nmr62\""
Sep 16 18:59:28.346055 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:28.346028    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j"]
Sep 16 18:59:28.427155 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:28.427119    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59c70509-13d1-4993-80f5-2ba78b244112-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j\" (UID: \"59c70509-13d1-4993-80f5-2ba78b244112\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j"
Sep 16 18:59:28.427774 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:28.427756    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59c70509-13d1-4993-80f5-2ba78b244112-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j\" (UID: \"59c70509-13d1-4993-80f5-2ba78b244112\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j"
Sep 16 18:59:28.633883 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:28.633853    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j"
Sep 16 18:59:28.938828 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:59:28.938781    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c70509_13d1_4993_80f5_2ba78b244112.slice/crio-73abb354256056cf4ce23e198e9ee0c7213fa9c8f9cb1bb5cc77ada5db06783c WatchSource:0}: Error finding container 73abb354256056cf4ce23e198e9ee0c7213fa9c8f9cb1bb5cc77ada5db06783c: Status 404 returned error can't find the container with id 73abb354256056cf4ce23e198e9ee0c7213fa9c8f9cb1bb5cc77ada5db06783c
Sep 16 18:59:28.940973 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:28.940935    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j"]
Sep 16 18:59:29.271917 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:29.271873    2578 patch_prober.go:28] interesting pod/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 16 18:59:29.272097 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:29.271938    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk" podUID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 16 18:59:29.842227 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:29.842158    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j" event={"ID":"59c70509-13d1-4993-80f5-2ba78b244112","Type":"ContainerStarted","Data":"d1d22f64340ae9da006729da237b57d033aea8895ef1a290898495b10c9fedba"}
Sep 16 18:59:29.842740 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:29.842292    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j" event={"ID":"59c70509-13d1-4993-80f5-2ba78b244112","Type":"ContainerStarted","Data":"73abb354256056cf4ce23e198e9ee0c7213fa9c8f9cb1bb5cc77ada5db06783c"}
Sep 16 18:59:29.866693 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:29.866629    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j" podStartSLOduration=1.866604564 podStartE2EDuration="1.866604564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:59:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:59:29.86414048 +0000 UTC m=+1197.651704638" watchObservedRunningTime="2026-09-16 18:59:29.866604564 +0000 UTC m=+1197.654168724"
Sep 16 18:59:30.271563 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:30.271524    2578 patch_prober.go:28] interesting pod/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 16 18:59:30.271744 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:30.271584    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk" podUID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 16 18:59:30.850499 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:30.850463    2578 generic.go:356] "Generic (PLEG): container finished" podID="59c70509-13d1-4993-80f5-2ba78b244112" containerID="d1d22f64340ae9da006729da237b57d033aea8895ef1a290898495b10c9fedba" exitCode=0
Sep 16 18:59:30.850885 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:30.850536    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j" event={"ID":"59c70509-13d1-4993-80f5-2ba78b244112","Type":"ContainerDied","Data":"d1d22f64340ae9da006729da237b57d033aea8895ef1a290898495b10c9fedba"}
Sep 16 18:59:31.275878 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:31.275846    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:59:31.276949 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:31.276925    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:59:31.994937 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:31.994914    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j"
Sep 16 18:59:32.061480 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:32.061448    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59c70509-13d1-4993-80f5-2ba78b244112-maas-api-service-ca\") pod \"59c70509-13d1-4993-80f5-2ba78b244112\" (UID: \"59c70509-13d1-4993-80f5-2ba78b244112\") "
Sep 16 18:59:32.061814 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:32.061791    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59c70509-13d1-4993-80f5-2ba78b244112-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "59c70509-13d1-4993-80f5-2ba78b244112" (UID: "59c70509-13d1-4993-80f5-2ba78b244112"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:59:32.162502 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:32.162416    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59c70509-13d1-4993-80f5-2ba78b244112-maas-api-service-ca\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:59:32.731848 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:32.731820    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 18:59:32.732013 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:32.731817    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 18:59:32.738160 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:32.738138    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 18:59:32.738312 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:32.738146    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 18:59:32.864742 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:32.864713    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j"
Sep 16 18:59:32.864923 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:32.864760    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j" event={"ID":"59c70509-13d1-4993-80f5-2ba78b244112","Type":"ContainerDied","Data":"73abb354256056cf4ce23e198e9ee0c7213fa9c8f9cb1bb5cc77ada5db06783c"}
Sep 16 18:59:32.864923 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:32.864836    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73abb354256056cf4ce23e198e9ee0c7213fa9c8f9cb1bb5cc77ada5db06783c"
Sep 16 18:59:41.493968 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:41.493930    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j"]
Sep 16 18:59:41.502660 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:41.502635    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-78ad8baa-xpn2j"]
Sep 16 18:59:41.683298 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:41.683264    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"]
Sep 16 18:59:41.683520 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:41.683497    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk" podUID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" containerName="istio-proxy" containerID="cri-o://2e965935314d2b4de02df1be88e737e6bd5b42ebe9fc625d84413c02d752d706" gracePeriod=30
Sep 16 18:59:42.563572 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.563537    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"]
Sep 16 18:59:42.563981 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.563968    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59c70509-13d1-4993-80f5-2ba78b244112" containerName="revoke-keys"
Sep 16 18:59:42.564031 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.563983    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c70509-13d1-4993-80f5-2ba78b244112" containerName="revoke-keys"
Sep 16 18:59:42.564073 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.564065    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="59c70509-13d1-4993-80f5-2ba78b244112" containerName="revoke-keys"
Sep 16 18:59:42.566966 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.566944    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.570491 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.570465    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8bed97ae-openshift-default-dockercfg-zxtv2\""
Sep 16 18:59:42.596470 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.596436    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"]
Sep 16 18:59:42.681486 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.681429    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4hsc\" (UniqueName: \"kubernetes.io/projected/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-kube-api-access-g4hsc\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.681641 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.681580    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-workload-socket\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.681641 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.681632    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-token\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.681728 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.681667    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-envoy\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.681728 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.681692    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istiod-ca-cert\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.681846 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.681825    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-credential-socket\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.681882 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.681867    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-workload-certs\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.681917 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.681892    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-podinfo\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.681956 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.681933    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-data\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.752197 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.751539    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59c70509-13d1-4993-80f5-2ba78b244112" path="/var/lib/kubelet/pods/59c70509-13d1-4993-80f5-2ba78b244112/volumes"
Sep 16 18:59:42.782681 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.782642    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-envoy\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.782681 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.782681    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istiod-ca-cert\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.782954 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.782742    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-credential-socket\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.782954 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.782768    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-workload-certs\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.782954 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.782796    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-podinfo\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.782954 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.782850    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-data\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.782954 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.782915    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4hsc\" (UniqueName: \"kubernetes.io/projected/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-kube-api-access-g4hsc\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.783281 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.782985    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-workload-socket\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.783281 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.783018    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-token\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.783281 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.783216    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-workload-certs\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.783281 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.783272    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-credential-socket\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.783575 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.783555    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istiod-ca-cert\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.783646 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.783591    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-workload-socket\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.783646 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.783608    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-data\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.785239 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.785211    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-envoy\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.785549 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.785519    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-podinfo\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.805961 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.805939    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-token\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.817466 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.817407    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4hsc\" (UniqueName: \"kubernetes.io/projected/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-kube-api-access-g4hsc\") pod \"e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") " pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:42.879888 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:42.879851    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:43.235909 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:43.235877    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"]
Sep 16 18:59:43.236023 ip-10-0-134-234 kubenswrapper[2578]: W0916 18:59:43.235880    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aaf86d8_45bd_4ff4_8aa6_391f4980cde0.slice/crio-1eb4ec93ceeab51b8d5055dd88a69b68d4bb0bc6f9928870290a1db235b5f11c WatchSource:0}: Error finding container 1eb4ec93ceeab51b8d5055dd88a69b68d4bb0bc6f9928870290a1db235b5f11c: Status 404 returned error can't find the container with id 1eb4ec93ceeab51b8d5055dd88a69b68d4bb0bc6f9928870290a1db235b5f11c
Sep 16 18:59:43.238370 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:43.238329    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:59:43.238476 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:43.238398    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:59:43.238476 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:43.238426    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 18:59:43.948157 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:43.948119    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm" event={"ID":"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0","Type":"ContainerStarted","Data":"2f1e5a5370cb01961ac9ef1316d59e92fff41a2b78c2df3313bba215f21e0f2c"}
Sep 16 18:59:43.948754 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:43.948723    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm" event={"ID":"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0","Type":"ContainerStarted","Data":"1eb4ec93ceeab51b8d5055dd88a69b68d4bb0bc6f9928870290a1db235b5f11c"}
Sep 16 18:59:43.992604 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:43.992557    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm" podStartSLOduration=1.9925400020000001 podStartE2EDuration="1.992540002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:59:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:59:43.992272047 +0000 UTC m=+1211.779836213" watchObservedRunningTime="2026-09-16 18:59:43.992540002 +0000 UTC m=+1211.780104157"
Sep 16 18:59:44.880791 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:44.880756    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:44.885661 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:44.885631    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:44.954849 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:44.954814    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:44.955930 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:44.955908    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 18:59:46.279160 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:46.279119    2578 patch_prober.go:28] interesting pod/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 18:59:46.279566 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:46.279208    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk" podUID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:59:46.951789 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:46.951718    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:59:46.967822 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:46.967787    2578 generic.go:356] "Generic (PLEG): container finished" podID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" containerID="2e965935314d2b4de02df1be88e737e6bd5b42ebe9fc625d84413c02d752d706" exitCode=0
Sep 16 18:59:46.967990 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:46.967838    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"
Sep 16 18:59:46.967990 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:46.967870    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk" event={"ID":"182d37f7-1a2a-4c6c-a20a-656fd0ee5354","Type":"ContainerDied","Data":"2e965935314d2b4de02df1be88e737e6bd5b42ebe9fc625d84413c02d752d706"}
Sep 16 18:59:46.967990 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:46.967946    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk" event={"ID":"182d37f7-1a2a-4c6c-a20a-656fd0ee5354","Type":"ContainerDied","Data":"11d8c626726e8675e0440779e4655908de03c8e08160a4940b610da08acf6bde"}
Sep 16 18:59:46.967990 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:46.967970    2578 scope.go:117] "RemoveContainer" containerID="2e965935314d2b4de02df1be88e737e6bd5b42ebe9fc625d84413c02d752d706"
Sep 16 18:59:46.978017 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:46.977988    2578 scope.go:117] "RemoveContainer" containerID="932e083655d975af0720b452f0236b95fd59eec437b400fba95b8deff65a021b"
Sep 16 18:59:46.986699 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:46.986677    2578 scope.go:117] "RemoveContainer" containerID="2e965935314d2b4de02df1be88e737e6bd5b42ebe9fc625d84413c02d752d706"
Sep 16 18:59:46.986991 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:59:46.986969    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e965935314d2b4de02df1be88e737e6bd5b42ebe9fc625d84413c02d752d706\": container with ID starting with 2e965935314d2b4de02df1be88e737e6bd5b42ebe9fc625d84413c02d752d706 not found: ID does not exist" containerID="2e965935314d2b4de02df1be88e737e6bd5b42ebe9fc625d84413c02d752d706"
Sep 16 18:59:46.987053 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:46.987000    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e965935314d2b4de02df1be88e737e6bd5b42ebe9fc625d84413c02d752d706"} err="failed to get container status \"2e965935314d2b4de02df1be88e737e6bd5b42ebe9fc625d84413c02d752d706\": rpc error: code = NotFound desc = could not find container \"2e965935314d2b4de02df1be88e737e6bd5b42ebe9fc625d84413c02d752d706\": container with ID starting with 2e965935314d2b4de02df1be88e737e6bd5b42ebe9fc625d84413c02d752d706 not found: ID does not exist"
Sep 16 18:59:46.987053 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:46.987020    2578 scope.go:117] "RemoveContainer" containerID="932e083655d975af0720b452f0236b95fd59eec437b400fba95b8deff65a021b"
Sep 16 18:59:46.987370 ip-10-0-134-234 kubenswrapper[2578]: E0916 18:59:46.987348    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"932e083655d975af0720b452f0236b95fd59eec437b400fba95b8deff65a021b\": container with ID starting with 932e083655d975af0720b452f0236b95fd59eec437b400fba95b8deff65a021b not found: ID does not exist" containerID="932e083655d975af0720b452f0236b95fd59eec437b400fba95b8deff65a021b"
Sep 16 18:59:46.987462 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:46.987379    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"932e083655d975af0720b452f0236b95fd59eec437b400fba95b8deff65a021b"} err="failed to get container status \"932e083655d975af0720b452f0236b95fd59eec437b400fba95b8deff65a021b\": rpc error: code = NotFound desc = could not find container \"932e083655d975af0720b452f0236b95fd59eec437b400fba95b8deff65a021b\": container with ID starting with 932e083655d975af0720b452f0236b95fd59eec437b400fba95b8deff65a021b not found: ID does not exist"
Sep 16 18:59:47.029965 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.029936    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istiod-ca-cert\") pod \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") "
Sep 16 18:59:47.030290 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.029986    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-workload-certs\") pod \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") "
Sep 16 18:59:47.030290 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030033    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjjh9\" (UniqueName: \"kubernetes.io/projected/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-kube-api-access-sjjh9\") pod \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") "
Sep 16 18:59:47.030290 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030065    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-workload-socket\") pod \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") "
Sep 16 18:59:47.030290 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030086    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-podinfo\") pod \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") "
Sep 16 18:59:47.030290 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030157    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-envoy\") pod \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") "
Sep 16 18:59:47.030290 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030219    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-data\") pod \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") "
Sep 16 18:59:47.030290 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030243    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-token\") pod \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") "
Sep 16 18:59:47.030676 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030306    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-credential-socket\") pod \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\" (UID: \"182d37f7-1a2a-4c6c-a20a-656fd0ee5354\") "
Sep 16 18:59:47.030676 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030410    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "182d37f7-1a2a-4c6c-a20a-656fd0ee5354" (UID: "182d37f7-1a2a-4c6c-a20a-656fd0ee5354"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:59:47.030676 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030447    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "182d37f7-1a2a-4c6c-a20a-656fd0ee5354" (UID: "182d37f7-1a2a-4c6c-a20a-656fd0ee5354"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:59:47.030676 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030452    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "182d37f7-1a2a-4c6c-a20a-656fd0ee5354" (UID: "182d37f7-1a2a-4c6c-a20a-656fd0ee5354"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:59:47.030676 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030667    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-data" (OuterVolumeSpecName: "istio-data") pod "182d37f7-1a2a-4c6c-a20a-656fd0ee5354" (UID: "182d37f7-1a2a-4c6c-a20a-656fd0ee5354"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:59:47.030931 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030799    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "182d37f7-1a2a-4c6c-a20a-656fd0ee5354" (UID: "182d37f7-1a2a-4c6c-a20a-656fd0ee5354"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:59:47.030988 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030934    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:59:47.030988 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030955    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:59:47.030988 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030970    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:59:47.030988 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030983    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:59:47.031127 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.030996    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:59:47.032509 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.032470    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-kube-api-access-sjjh9" (OuterVolumeSpecName: "kube-api-access-sjjh9") pod "182d37f7-1a2a-4c6c-a20a-656fd0ee5354" (UID: "182d37f7-1a2a-4c6c-a20a-656fd0ee5354"). InnerVolumeSpecName "kube-api-access-sjjh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:59:47.032616 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.032509    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "182d37f7-1a2a-4c6c-a20a-656fd0ee5354" (UID: "182d37f7-1a2a-4c6c-a20a-656fd0ee5354"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 18:59:47.032836 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.032812    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-token" (OuterVolumeSpecName: "istio-token") pod "182d37f7-1a2a-4c6c-a20a-656fd0ee5354" (UID: "182d37f7-1a2a-4c6c-a20a-656fd0ee5354"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:59:47.032957 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.032924    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "182d37f7-1a2a-4c6c-a20a-656fd0ee5354" (UID: "182d37f7-1a2a-4c6c-a20a-656fd0ee5354"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:59:47.132596 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.132561    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:59:47.132596 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.132593    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:59:47.132596 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.132604    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjjh9\" (UniqueName: \"kubernetes.io/projected/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-kube-api-access-sjjh9\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:59:47.132596 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.132614    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/182d37f7-1a2a-4c6c-a20a-656fd0ee5354-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 18:59:47.299587 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.299549    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"]
Sep 16 18:59:47.314866 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:47.314834    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-78ad8baa-gw-openshift-default-fc4568876-gf6jk"]
Sep 16 18:59:48.710755 ip-10-0-134-234 kubenswrapper[2578]: I0916 18:59:48.710719    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" path="/var/lib/kubelet/pods/182d37f7-1a2a-4c6c-a20a-656fd0ee5354/volumes"
Sep 16 19:00:00.189082 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.189045    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2efa6f-29826420-vlv46"]
Sep 16 19:00:00.189522 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.189507    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" containerName="istio-proxy"
Sep 16 19:00:00.189522 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.189523    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" containerName="istio-proxy"
Sep 16 19:00:00.189596 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.189545    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" containerName="istio-proxy"
Sep 16 19:00:00.189596 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.189551    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" containerName="istio-proxy"
Sep 16 19:00:00.189666 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.189639    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" containerName="istio-proxy"
Sep 16 19:00:00.189666 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.189648    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="182d37f7-1a2a-4c6c-a20a-656fd0ee5354" containerName="istio-proxy"
Sep 16 19:00:00.192530 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.192513    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2efa6f-29826420-vlv46"
Sep 16 19:00:00.195917 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.195891    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nmr62\""
Sep 16 19:00:00.202877 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.202847    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8bed97ae-29826420-8vn6t"]
Sep 16 19:00:00.203489 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.203470    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2efa6f-29826420-vlv46"
Sep 16 19:00:00.205642 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.205400    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8bed97ae-29826420-8vn6t"
Sep 16 19:00:00.212630 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.212207    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826420-w47t2"]
Sep 16 19:00:00.216827 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.216401    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826420-w47t2"
Sep 16 19:00:00.217301 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.217265    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2efa6f-29826420-vlv46"]
Sep 16 19:00:00.227508 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.227408    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8bed97ae-29826420-8vn6t"]
Sep 16 19:00:00.235017 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.234921    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826420-w47t2"]
Sep 16 19:00:00.262212 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.253415    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8bed97ae-29826420-8vn6t"
Sep 16 19:00:00.282535 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.282494    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a45a9-29826420-2k9sr"]
Sep 16 19:00:00.286213 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.286059    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a45a9-29826420-2k9sr"
Sep 16 19:00:00.286300 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.286210    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a45a9-29826420-2k9sr"]
Sep 16 19:00:00.316649 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.316380    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826420-w47t2"
Sep 16 19:00:00.328540 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.328350    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a45a9-29826420-2k9sr"
Sep 16 19:00:00.534130 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.534100    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2efa6f-29826420-vlv46"]
Sep 16 19:00:00.534751 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:00:00.534728    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode70a1682_a094_4eb5_9ce9_b84ddbfd1ea5.slice/crio-bc4e30073a76a99eb3abfd290f6fd6d66f143bf2bb0a714613b3c149f3deca04 WatchSource:0}: Error finding container bc4e30073a76a99eb3abfd290f6fd6d66f143bf2bb0a714613b3c149f3deca04: Status 404 returned error can't find the container with id bc4e30073a76a99eb3abfd290f6fd6d66f143bf2bb0a714613b3c149f3deca04
Sep 16 19:00:00.623427 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.623398    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8bed97ae-29826420-8vn6t"]
Sep 16 19:00:00.624769 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:00:00.624740    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda875c7a_d297_4a2a_a448_24335e170e49.slice/crio-d1f83c9c6f580684f9e791abec72766e32f45b6c42c02e81cde0c97bc7d6d7e9 WatchSource:0}: Error finding container d1f83c9c6f580684f9e791abec72766e32f45b6c42c02e81cde0c97bc7d6d7e9: Status 404 returned error can't find the container with id d1f83c9c6f580684f9e791abec72766e32f45b6c42c02e81cde0c97bc7d6d7e9
Sep 16 19:00:00.674502 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.674477    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826420-w47t2"]
Sep 16 19:00:00.685462 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:00:00.685433    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbc9ab59_350e_408a_82f1_b6e59404c51f.slice/crio-7dcbc6ed7c5d660b117577e0bb8de7e90d90cffd72dfa0091f330bb6eb381446 WatchSource:0}: Error finding container 7dcbc6ed7c5d660b117577e0bb8de7e90d90cffd72dfa0091f330bb6eb381446: Status 404 returned error can't find the container with id 7dcbc6ed7c5d660b117577e0bb8de7e90d90cffd72dfa0091f330bb6eb381446
Sep 16 19:00:00.704712 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:00:00.704683    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6fc2bbf_a9f4_465b_9c81_acc8340e3111.slice/crio-ceada8abb101e493752eb4a70ab576e2fc1a76066b93eb534ebd83049963081e WatchSource:0}: Error finding container ceada8abb101e493752eb4a70ab576e2fc1a76066b93eb534ebd83049963081e: Status 404 returned error can't find the container with id ceada8abb101e493752eb4a70ab576e2fc1a76066b93eb534ebd83049963081e
Sep 16 19:00:00.711202 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:00.711162    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a45a9-29826420-2k9sr"]
Sep 16 19:00:01.069250 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:01.069110    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826420-w47t2" event={"ID":"dbc9ab59-350e-408a-82f1-b6e59404c51f","Type":"ContainerStarted","Data":"7dcbc6ed7c5d660b117577e0bb8de7e90d90cffd72dfa0091f330bb6eb381446"}
Sep 16 19:00:01.071412 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:01.071378    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2efa6f-29826420-vlv46" event={"ID":"e70a1682-a094-4eb5-9ce9-b84ddbfd1ea5","Type":"ContainerStarted","Data":"2a8c4d24098de8c570897e751042510937886704d0a90cb174fa4e5152cf8fe0"}
Sep 16 19:00:01.071578 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:01.071432    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2efa6f-29826420-vlv46" event={"ID":"e70a1682-a094-4eb5-9ce9-b84ddbfd1ea5","Type":"ContainerStarted","Data":"bc4e30073a76a99eb3abfd290f6fd6d66f143bf2bb0a714613b3c149f3deca04"}
Sep 16 19:00:01.072814 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:01.072784    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a45a9-29826420-2k9sr" event={"ID":"d6fc2bbf-a9f4-465b-9c81-acc8340e3111","Type":"ContainerStarted","Data":"ceada8abb101e493752eb4a70ab576e2fc1a76066b93eb534ebd83049963081e"}
Sep 16 19:00:01.073938 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:01.073914    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8bed97ae-29826420-8vn6t" event={"ID":"da875c7a-d297-4a2a-a448-24335e170e49","Type":"ContainerStarted","Data":"d1f83c9c6f580684f9e791abec72766e32f45b6c42c02e81cde0c97bc7d6d7e9"}
Sep 16 19:00:01.090763 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:01.090719    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2efa6f-29826420-vlv46" podStartSLOduration=1.090705715 podStartE2EDuration="1.090705715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:00:01.087063783 +0000 UTC m=+1228.874627938" watchObservedRunningTime="2026-09-16 19:00:01.090705715 +0000 UTC m=+1228.878269869"
Sep 16 19:00:02.082187 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:02.082150    2578 generic.go:356] "Generic (PLEG): container finished" podID="da875c7a-d297-4a2a-a448-24335e170e49" containerID="77a47b8d018d872b7424c615daf1aaf15ed56d143293022483e29d476ab4dd9a" exitCode=0
Sep 16 19:00:02.082583 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:02.082208    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8bed97ae-29826420-8vn6t" event={"ID":"da875c7a-d297-4a2a-a448-24335e170e49","Type":"ContainerDied","Data":"77a47b8d018d872b7424c615daf1aaf15ed56d143293022483e29d476ab4dd9a"}
Sep 16 19:00:02.084020 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:02.083998    2578 generic.go:356] "Generic (PLEG): container finished" podID="dbc9ab59-350e-408a-82f1-b6e59404c51f" containerID="a6a9f27f290b1af5c8b8c06bc9817ba9171fe3d7486afd0b60af8c426c27939d" exitCode=0
Sep 16 19:00:02.084131 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:02.084067    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826420-w47t2" event={"ID":"dbc9ab59-350e-408a-82f1-b6e59404c51f","Type":"ContainerDied","Data":"a6a9f27f290b1af5c8b8c06bc9817ba9171fe3d7486afd0b60af8c426c27939d"}
Sep 16 19:00:02.085509 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:02.085488    2578 generic.go:356] "Generic (PLEG): container finished" podID="e70a1682-a094-4eb5-9ce9-b84ddbfd1ea5" containerID="2a8c4d24098de8c570897e751042510937886704d0a90cb174fa4e5152cf8fe0" exitCode=0
Sep 16 19:00:02.085615 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:02.085568    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2efa6f-29826420-vlv46" event={"ID":"e70a1682-a094-4eb5-9ce9-b84ddbfd1ea5","Type":"ContainerDied","Data":"2a8c4d24098de8c570897e751042510937886704d0a90cb174fa4e5152cf8fe0"}
Sep 16 19:00:02.086990 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:02.086971    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a45a9-29826420-2k9sr" event={"ID":"d6fc2bbf-a9f4-465b-9c81-acc8340e3111","Type":"ContainerStarted","Data":"8f4bf74569e9dec67bb8750f7e18a98f5177a557ca3c27b856422a15f096b951"}
Sep 16 19:00:02.145139 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:02.145071    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a45a9-29826420-2k9sr" podStartSLOduration=2.145026825 podStartE2EDuration="2.145026825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:00:02.136992787 +0000 UTC m=+1229.924556969" watchObservedRunningTime="2026-09-16 19:00:02.145026825 +0000 UTC m=+1229.932590981"
Sep 16 19:00:03.095122 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:03.095085    2578 generic.go:356] "Generic (PLEG): container finished" podID="d6fc2bbf-a9f4-465b-9c81-acc8340e3111" containerID="8f4bf74569e9dec67bb8750f7e18a98f5177a557ca3c27b856422a15f096b951" exitCode=0
Sep 16 19:00:03.095567 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:03.095197    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a45a9-29826420-2k9sr" event={"ID":"d6fc2bbf-a9f4-465b-9c81-acc8340e3111","Type":"ContainerDied","Data":"8f4bf74569e9dec67bb8750f7e18a98f5177a557ca3c27b856422a15f096b951"}
Sep 16 19:00:03.375717 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:03.375684    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8bed97ae-29826420-8vn6t"
Sep 16 19:00:03.378985 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:03.378963    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826420-w47t2"
Sep 16 19:00:03.382790 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:03.382772    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2efa6f-29826420-vlv46"
Sep 16 19:00:04.102635 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:04.102605    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826420-w47t2"
Sep 16 19:00:04.102635 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:04.102627    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826420-w47t2" event={"ID":"dbc9ab59-350e-408a-82f1-b6e59404c51f","Type":"ContainerDied","Data":"7dcbc6ed7c5d660b117577e0bb8de7e90d90cffd72dfa0091f330bb6eb381446"}
Sep 16 19:00:04.103050 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:04.102680    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7dcbc6ed7c5d660b117577e0bb8de7e90d90cffd72dfa0091f330bb6eb381446"
Sep 16 19:00:04.104234 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:04.104216    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2efa6f-29826420-vlv46" event={"ID":"e70a1682-a094-4eb5-9ce9-b84ddbfd1ea5","Type":"ContainerDied","Data":"bc4e30073a76a99eb3abfd290f6fd6d66f143bf2bb0a714613b3c149f3deca04"}
Sep 16 19:00:04.104302 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:04.104256    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc4e30073a76a99eb3abfd290f6fd6d66f143bf2bb0a714613b3c149f3deca04"
Sep 16 19:00:04.104302 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:04.104298    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2efa6f-29826420-vlv46"
Sep 16 19:00:04.105763 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:04.105739    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8bed97ae-29826420-8vn6t"
Sep 16 19:00:04.105857 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:04.105764    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8bed97ae-29826420-8vn6t" event={"ID":"da875c7a-d297-4a2a-a448-24335e170e49","Type":"ContainerDied","Data":"d1f83c9c6f580684f9e791abec72766e32f45b6c42c02e81cde0c97bc7d6d7e9"}
Sep 16 19:00:04.105857 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:04.105813    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1f83c9c6f580684f9e791abec72766e32f45b6c42c02e81cde0c97bc7d6d7e9"
Sep 16 19:00:04.295539 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:04.295514    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a45a9-29826420-2k9sr"
Sep 16 19:00:05.114137 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:05.114105    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a45a9-29826420-2k9sr"
Sep 16 19:00:05.114137 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:05.114128    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a45a9-29826420-2k9sr" event={"ID":"d6fc2bbf-a9f4-465b-9c81-acc8340e3111","Type":"ContainerDied","Data":"ceada8abb101e493752eb4a70ab576e2fc1a76066b93eb534ebd83049963081e"}
Sep 16 19:00:05.114592 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:05.114222    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ceada8abb101e493752eb4a70ab576e2fc1a76066b93eb534ebd83049963081e"
Sep 16 19:00:08.135113 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:08.135078    2578 generic.go:356] "Generic (PLEG): container finished" podID="1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" containerID="2f1e5a5370cb01961ac9ef1316d59e92fff41a2b78c2df3313bba215f21e0f2c" exitCode=0
Sep 16 19:00:08.135622 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:08.135150    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm" event={"ID":"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0","Type":"ContainerDied","Data":"2f1e5a5370cb01961ac9ef1316d59e92fff41a2b78c2df3313bba215f21e0f2c"}
Sep 16 19:00:08.135692 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:08.135677    2578 scope.go:117] "RemoveContainer" containerID="2f1e5a5370cb01961ac9ef1316d59e92fff41a2b78c2df3313bba215f21e0f2c"
Sep 16 19:00:08.136884 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:08.136852    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:00:08.136936 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:08.136924    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:00:08.136985 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:08.136956    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:00:09.145740 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:09.145701    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm" event={"ID":"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0","Type":"ContainerStarted","Data":"1d701d23c2059118304f77c02cabc55bd23b94c4752b674a6552b44972bbdbae"}
Sep 16 19:00:09.881071 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:09.881035    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 19:00:09.881780 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:09.881754    2578 patch_prober.go:28] interesting pod/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 16 19:00:09.881894 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:09.881810    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm" podUID="1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 16 19:00:10.880789 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:10.880747    2578 patch_prober.go:28] interesting pod/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 16 19:00:10.881219 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:10.880821    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm" podUID="1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 16 19:00:11.880494 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:11.880451    2578 patch_prober.go:28] interesting pod/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 16 19:00:11.880690 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:11.880520    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm" podUID="1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 16 19:00:12.880239 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:12.880201    2578 patch_prober.go:28] interesting pod/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 16 19:00:12.880697 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:12.880271    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm" podUID="1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 16 19:00:13.880857 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:13.880820    2578 patch_prober.go:28] interesting pod/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 16 19:00:13.881348 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:13.880888    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm" podUID="1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 16 19:00:14.955671 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:14.955633    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 19:00:14.968746 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:14.968717    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 19:00:15.207668 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:15.207587    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 19:00:16.531356 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.531323    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j"]
Sep 16 19:00:16.531882 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.531863    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da875c7a-d297-4a2a-a448-24335e170e49" containerName="cleanup"
Sep 16 19:00:16.531928 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.531884    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="da875c7a-d297-4a2a-a448-24335e170e49" containerName="cleanup"
Sep 16 19:00:16.531928 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.531899    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e70a1682-a094-4eb5-9ce9-b84ddbfd1ea5" containerName="cleanup"
Sep 16 19:00:16.531928 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.531904    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e70a1682-a094-4eb5-9ce9-b84ddbfd1ea5" containerName="cleanup"
Sep 16 19:00:16.531928 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.531925    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbc9ab59-350e-408a-82f1-b6e59404c51f" containerName="cleanup"
Sep 16 19:00:16.532056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.531931    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbc9ab59-350e-408a-82f1-b6e59404c51f" containerName="cleanup"
Sep 16 19:00:16.532056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.531937    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6fc2bbf-a9f4-465b-9c81-acc8340e3111" containerName="cleanup"
Sep 16 19:00:16.532056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.531942    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6fc2bbf-a9f4-465b-9c81-acc8340e3111" containerName="cleanup"
Sep 16 19:00:16.532056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.532033    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbc9ab59-350e-408a-82f1-b6e59404c51f" containerName="cleanup"
Sep 16 19:00:16.532056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.532041    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e70a1682-a094-4eb5-9ce9-b84ddbfd1ea5" containerName="cleanup"
Sep 16 19:00:16.532056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.532051    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="da875c7a-d297-4a2a-a448-24335e170e49" containerName="cleanup"
Sep 16 19:00:16.532056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.532058    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6fc2bbf-a9f4-465b-9c81-acc8340e3111" containerName="cleanup"
Sep 16 19:00:16.534978 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.534959    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j"
Sep 16 19:00:16.539967 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.539944    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nmr62\""
Sep 16 19:00:16.548605 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.548577    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j"]
Sep 16 19:00:16.648852 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.648787    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72831d7f-77fc-4b77-a1b6-e4fe6d37bed9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j\" (UID: \"72831d7f-77fc-4b77-a1b6-e4fe6d37bed9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j"
Sep 16 19:00:16.749794 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.749748    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72831d7f-77fc-4b77-a1b6-e4fe6d37bed9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j\" (UID: \"72831d7f-77fc-4b77-a1b6-e4fe6d37bed9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j"
Sep 16 19:00:16.750492 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.750469    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72831d7f-77fc-4b77-a1b6-e4fe6d37bed9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j\" (UID: \"72831d7f-77fc-4b77-a1b6-e4fe6d37bed9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j"
Sep 16 19:00:16.847242 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:16.847127    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j"
Sep 16 19:00:17.131897 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:17.131868    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j"]
Sep 16 19:00:17.133430 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:00:17.133390    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72831d7f_77fc_4b77_a1b6_e4fe6d37bed9.slice/crio-cab500d408db2933b22a4f2dee850d234dc876eb92c92cd2035b6c9554e30adc WatchSource:0}: Error finding container cab500d408db2933b22a4f2dee850d234dc876eb92c92cd2035b6c9554e30adc: Status 404 returned error can't find the container with id cab500d408db2933b22a4f2dee850d234dc876eb92c92cd2035b6c9554e30adc
Sep 16 19:00:17.223550 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:17.223509    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j" event={"ID":"72831d7f-77fc-4b77-a1b6-e4fe6d37bed9","Type":"ContainerStarted","Data":"cab500d408db2933b22a4f2dee850d234dc876eb92c92cd2035b6c9554e30adc"}
Sep 16 19:00:18.231390 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:18.231353    2578 generic.go:356] "Generic (PLEG): container finished" podID="72831d7f-77fc-4b77-a1b6-e4fe6d37bed9" containerID="fc779e6146893edf719a2e9ee6b29ec3e7dc3cec4c9d72afa76b257e147e5486" exitCode=0
Sep 16 19:00:18.231778 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:18.231432    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j" event={"ID":"72831d7f-77fc-4b77-a1b6-e4fe6d37bed9","Type":"ContainerDied","Data":"fc779e6146893edf719a2e9ee6b29ec3e7dc3cec4c9d72afa76b257e147e5486"}
Sep 16 19:00:19.384151 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:19.384127    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j"
Sep 16 19:00:19.480533 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:19.480500    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72831d7f-77fc-4b77-a1b6-e4fe6d37bed9-maas-api-service-ca\") pod \"72831d7f-77fc-4b77-a1b6-e4fe6d37bed9\" (UID: \"72831d7f-77fc-4b77-a1b6-e4fe6d37bed9\") "
Sep 16 19:00:19.480851 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:19.480828    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72831d7f-77fc-4b77-a1b6-e4fe6d37bed9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "72831d7f-77fc-4b77-a1b6-e4fe6d37bed9" (UID: "72831d7f-77fc-4b77-a1b6-e4fe6d37bed9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:00:19.582021 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:19.581938    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72831d7f-77fc-4b77-a1b6-e4fe6d37bed9-maas-api-service-ca\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:00:20.245886 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:20.245858    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j"
Sep 16 19:00:20.246064 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:20.245894    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j" event={"ID":"72831d7f-77fc-4b77-a1b6-e4fe6d37bed9","Type":"ContainerDied","Data":"cab500d408db2933b22a4f2dee850d234dc876eb92c92cd2035b6c9554e30adc"}
Sep 16 19:00:20.246064 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:20.245940    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cab500d408db2933b22a4f2dee850d234dc876eb92c92cd2035b6c9554e30adc"
Sep 16 19:00:23.103863 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.103825    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"]
Sep 16 19:00:23.104342 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.104327    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72831d7f-77fc-4b77-a1b6-e4fe6d37bed9" containerName="revoke-keys"
Sep 16 19:00:23.104342 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.104342    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="72831d7f-77fc-4b77-a1b6-e4fe6d37bed9" containerName="revoke-keys"
Sep 16 19:00:23.104433 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.104423    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="72831d7f-77fc-4b77-a1b6-e4fe6d37bed9" containerName="revoke-keys"
Sep 16 19:00:23.107837 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.107819    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.112160 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.112136    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-ec32e7\"/\"kube-root-ca.crt\""
Sep 16 19:00:23.112326 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.112139    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-ec32e7\"/\"test-model-ec32e7-kserve-self-signed-certs\""
Sep 16 19:00:23.112326 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.112227    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-ec32e7\"/\"openshift-service-ca.crt\""
Sep 16 19:00:23.112326 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.112139    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-ec32e7\"/\"default-dockercfg-5m7tw\""
Sep 16 19:00:23.124161 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.124131    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"]
Sep 16 19:00:23.219810 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.219773    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-dshm\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.220007 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.219820    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-home\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.220007 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.219841    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-tmp-dir\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.220007 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.219962    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67mgm\" (UniqueName: \"kubernetes.io/projected/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-kube-api-access-67mgm\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.220194 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.220059    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-model-cache\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.220194 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.220139    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-tls-certs\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.321278 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.321227    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-tls-certs\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.321484 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.321335    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-dshm\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.321484 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.321378    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-home\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.321484 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.321404    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-tmp-dir\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.321484 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.321453    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67mgm\" (UniqueName: \"kubernetes.io/projected/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-kube-api-access-67mgm\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.321648 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.321512    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-model-cache\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.321960 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.321922    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-tmp-dir\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.321960 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.321946    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-model-cache\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.322122 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.322000    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-home\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.323707 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.323685    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-dshm\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.323962 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.323945    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-tls-certs\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.334682 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.334655    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67mgm\" (UniqueName: \"kubernetes.io/projected/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-kube-api-access-67mgm\") pod \"test-model-ec32e7-kserve-7ff6d69c9-zdlqc\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") " pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.418230 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.418167    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:23.709046 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:23.709016    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"]
Sep 16 19:00:23.709892 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:00:23.709866    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a51d474_0d49_4e6b_b5d0_e4c5bb59a0f5.slice/crio-f4ce502e1ae79cc633b0e395c41f55cb0f807415e1ae80b2bf1a351a9e667e96 WatchSource:0}: Error finding container f4ce502e1ae79cc633b0e395c41f55cb0f807415e1ae80b2bf1a351a9e667e96: Status 404 returned error can't find the container with id f4ce502e1ae79cc633b0e395c41f55cb0f807415e1ae80b2bf1a351a9e667e96
Sep 16 19:00:24.274188 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:24.274139    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc" event={"ID":"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5","Type":"ContainerStarted","Data":"fc00e541215231c75732f553931f753c2cbc9779712ce03ec9f89124c9a46217"}
Sep 16 19:00:24.274653 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:24.274222    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc" event={"ID":"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5","Type":"ContainerStarted","Data":"f4ce502e1ae79cc633b0e395c41f55cb0f807415e1ae80b2bf1a351a9e667e96"}
Sep 16 19:00:24.313635 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:24.313580    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc" podStartSLOduration=1.3135596999999999 podStartE2EDuration="1.3135597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:00:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:00:24.308840301 +0000 UTC m=+1252.096404473" watchObservedRunningTime="2026-09-16 19:00:24.3135597 +0000 UTC m=+1252.101123858"
Sep 16 19:00:28.076560 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:28.076479    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8bed97ae-29826420-8vn6t"]
Sep 16 19:00:28.081300 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:28.081272    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-8bed97ae-29826420-8vn6t"]
Sep 16 19:00:28.545886 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:28.545850    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j"]
Sep 16 19:00:28.560948 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:28.560911    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bed97ae-9b78j"]
Sep 16 19:00:28.710960 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:28.710923    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72831d7f-77fc-4b77-a1b6-e4fe6d37bed9" path="/var/lib/kubelet/pods/72831d7f-77fc-4b77-a1b6-e4fe6d37bed9/volumes"
Sep 16 19:00:28.711282 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:28.711267    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da875c7a-d297-4a2a-a448-24335e170e49" path="/var/lib/kubelet/pods/da875c7a-d297-4a2a-a448-24335e170e49/volumes"
Sep 16 19:00:29.868588 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:29.868557    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"]
Sep 16 19:00:29.869006 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:29.868778    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm" podUID="1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" containerName="istio-proxy" containerID="cri-o://1d701d23c2059118304f77c02cabc55bd23b94c4752b674a6552b44972bbdbae" gracePeriod=30
Sep 16 19:00:30.209672 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:30.209575    2578 patch_prober.go:28] interesting pod/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 16 19:00:30.209672 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:30.209638    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm" podUID="1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:00:30.826565 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:30.826512    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"]
Sep 16 19:00:30.833422 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:30.833393    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:30.840004 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:30.839980    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-96b42948-openshift-default-dockercfg-c55hl\""
Sep 16 19:00:30.862037 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:30.862005    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"]
Sep 16 19:00:30.920165 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:30.918771    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-token\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:30.920165 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:30.918839    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-podinfo\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:30.920165 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:30.918897    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-credential-socket\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:30.920165 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:30.918931    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-workload-certs\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:30.920165 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:30.919003    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-workload-socket\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:30.920165 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:30.919033    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xfqz\" (UniqueName: \"kubernetes.io/projected/c382fd09-1338-4dcd-a662-36d4483ca83f-kube-api-access-7xfqz\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:30.920165 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:30.919065    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-envoy\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:30.920165 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:30.919101    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-data\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:30.920165 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:30.919157    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c382fd09-1338-4dcd-a662-36d4483ca83f-istiod-ca-cert\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.020780 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.020743    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-credential-socket\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.020994 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.020793    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-workload-certs\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.020994 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.020874    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-workload-socket\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.020994 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.020905    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xfqz\" (UniqueName: \"kubernetes.io/projected/c382fd09-1338-4dcd-a662-36d4483ca83f-kube-api-access-7xfqz\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.020994 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.020934    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-envoy\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.020994 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.020971    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-data\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.021320 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.021033    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c382fd09-1338-4dcd-a662-36d4483ca83f-istiod-ca-cert\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.021320 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.021076    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-token\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.021320 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.021122    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-podinfo\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.021320 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.021269    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-credential-socket\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.021560 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.021316    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-workload-certs\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.021560 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.021544    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-workload-socket\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.021741 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.021715    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-data\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.021867 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.021846    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c382fd09-1338-4dcd-a662-36d4483ca83f-istiod-ca-cert\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.023715 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.023697    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-podinfo\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.023810 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.023758    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-envoy\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.052387 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.052359    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-token\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.056531 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.056494    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xfqz\" (UniqueName: \"kubernetes.io/projected/c382fd09-1338-4dcd-a662-36d4483ca83f-kube-api-access-7xfqz\") pod \"e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") " pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.144513 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.144472    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:31.453222 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.453196    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"]
Sep 16 19:00:31.454893 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:00:31.454862    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc382fd09_1338_4dcd_a662_36d4483ca83f.slice/crio-73f51d7226eb052296b64deaa4b3a0c56f9fbc24a9a8c497c8b2af2dd8fa3a91 WatchSource:0}: Error finding container 73f51d7226eb052296b64deaa4b3a0c56f9fbc24a9a8c497c8b2af2dd8fa3a91: Status 404 returned error can't find the container with id 73f51d7226eb052296b64deaa4b3a0c56f9fbc24a9a8c497c8b2af2dd8fa3a91
Sep 16 19:00:31.457283 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.457244    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:00:31.457425 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.457334    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:00:31.457425 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:31.457388    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:00:32.334971 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:32.334927    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6" event={"ID":"c382fd09-1338-4dcd-a662-36d4483ca83f","Type":"ContainerStarted","Data":"6365410149beccc4dc79ad0455a017d6a23e48cfbef7663ad41c7294b2d1f91b"}
Sep 16 19:00:32.335593 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:32.335003    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6" event={"ID":"c382fd09-1338-4dcd-a662-36d4483ca83f","Type":"ContainerStarted","Data":"73f51d7226eb052296b64deaa4b3a0c56f9fbc24a9a8c497c8b2af2dd8fa3a91"}
Sep 16 19:00:32.375055 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:32.375001    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6" podStartSLOduration=2.37498521 podStartE2EDuration="2.37498521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:00:32.374858326 +0000 UTC m=+1260.162422481" watchObservedRunningTime="2026-09-16 19:00:32.37498521 +0000 UTC m=+1260.162549364"
Sep 16 19:00:33.145007 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:33.144972    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:33.151089 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:33.151061    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:33.342157 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:33.342123    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:33.343077 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:33.343057    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:00:33.419263 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:33.419163    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:33.419263 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:33.419237    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:33.432649 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:33.432617    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:34.360271 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:34.360236    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:00:35.161648 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.161581    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 19:00:35.269558 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.269523    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-credential-socket\") pod \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") "
Sep 16 19:00:35.269558 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.269577    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-podinfo\") pod \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") "
Sep 16 19:00:35.269836 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.269652    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-workload-socket\") pod \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") "
Sep 16 19:00:35.269836 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.269683    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-workload-certs\") pod \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") "
Sep 16 19:00:35.269836 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.269715    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4hsc\" (UniqueName: \"kubernetes.io/projected/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-kube-api-access-g4hsc\") pod \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") "
Sep 16 19:00:35.269836 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.269753    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-envoy\") pod \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") "
Sep 16 19:00:35.269836 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.269784    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-token\") pod \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") "
Sep 16 19:00:35.270086 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.269843    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-data\") pod \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") "
Sep 16 19:00:35.270086 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.269906    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istiod-ca-cert\") pod \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\" (UID: \"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0\") "
Sep 16 19:00:35.270372 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.270339    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" (UID: "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:00:35.270534 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.270509    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" (UID: "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:00:35.270651 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.270552    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" (UID: "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:00:35.270798 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.270757    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-data" (OuterVolumeSpecName: "istio-data") pod "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" (UID: "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:00:35.270798 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.270763    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" (UID: "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:00:35.272671 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.272647    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-token" (OuterVolumeSpecName: "istio-token") pod "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" (UID: "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:00:35.272821 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.272797    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-kube-api-access-g4hsc" (OuterVolumeSpecName: "kube-api-access-g4hsc") pod "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" (UID: "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0"). InnerVolumeSpecName "kube-api-access-g4hsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:00:35.272878 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.272812    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" (UID: "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:00:35.272878 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.272830    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" (UID: "1aaf86d8-45bd-4ff4-8aa6-391f4980cde0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:00:35.357200 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.357137    2578 generic.go:356] "Generic (PLEG): container finished" podID="1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" containerID="1d701d23c2059118304f77c02cabc55bd23b94c4752b674a6552b44972bbdbae" exitCode=0
Sep 16 19:00:35.357371 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.357221    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"
Sep 16 19:00:35.357371 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.357222    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm" event={"ID":"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0","Type":"ContainerDied","Data":"1d701d23c2059118304f77c02cabc55bd23b94c4752b674a6552b44972bbdbae"}
Sep 16 19:00:35.357371 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.357280    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm" event={"ID":"1aaf86d8-45bd-4ff4-8aa6-391f4980cde0","Type":"ContainerDied","Data":"1eb4ec93ceeab51b8d5055dd88a69b68d4bb0bc6f9928870290a1db235b5f11c"}
Sep 16 19:00:35.357371 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.357308    2578 scope.go:117] "RemoveContainer" containerID="1d701d23c2059118304f77c02cabc55bd23b94c4752b674a6552b44972bbdbae"
Sep 16 19:00:35.368306 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.368289    2578 scope.go:117] "RemoveContainer" containerID="2f1e5a5370cb01961ac9ef1316d59e92fff41a2b78c2df3313bba215f21e0f2c"
Sep 16 19:00:35.371842 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.371819    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:00:35.371842 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.371841    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4hsc\" (UniqueName: \"kubernetes.io/projected/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-kube-api-access-g4hsc\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:00:35.372010 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.371851    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:00:35.372010 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.371860    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:00:35.372010 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.371867    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:00:35.372010 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.371877    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:00:35.372010 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.371885    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:00:35.372010 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.371892    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:00:35.372010 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.371900    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:00:35.378011 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.377988    2578 scope.go:117] "RemoveContainer" containerID="1d701d23c2059118304f77c02cabc55bd23b94c4752b674a6552b44972bbdbae"
Sep 16 19:00:35.378362 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:00:35.378340    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d701d23c2059118304f77c02cabc55bd23b94c4752b674a6552b44972bbdbae\": container with ID starting with 1d701d23c2059118304f77c02cabc55bd23b94c4752b674a6552b44972bbdbae not found: ID does not exist" containerID="1d701d23c2059118304f77c02cabc55bd23b94c4752b674a6552b44972bbdbae"
Sep 16 19:00:35.378435 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.378372    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d701d23c2059118304f77c02cabc55bd23b94c4752b674a6552b44972bbdbae"} err="failed to get container status \"1d701d23c2059118304f77c02cabc55bd23b94c4752b674a6552b44972bbdbae\": rpc error: code = NotFound desc = could not find container \"1d701d23c2059118304f77c02cabc55bd23b94c4752b674a6552b44972bbdbae\": container with ID starting with 1d701d23c2059118304f77c02cabc55bd23b94c4752b674a6552b44972bbdbae not found: ID does not exist"
Sep 16 19:00:35.378435 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.378392    2578 scope.go:117] "RemoveContainer" containerID="2f1e5a5370cb01961ac9ef1316d59e92fff41a2b78c2df3313bba215f21e0f2c"
Sep 16 19:00:35.378670 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:00:35.378652    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f1e5a5370cb01961ac9ef1316d59e92fff41a2b78c2df3313bba215f21e0f2c\": container with ID starting with 2f1e5a5370cb01961ac9ef1316d59e92fff41a2b78c2df3313bba215f21e0f2c not found: ID does not exist" containerID="2f1e5a5370cb01961ac9ef1316d59e92fff41a2b78c2df3313bba215f21e0f2c"
Sep 16 19:00:35.378726 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.378672    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f1e5a5370cb01961ac9ef1316d59e92fff41a2b78c2df3313bba215f21e0f2c"} err="failed to get container status \"2f1e5a5370cb01961ac9ef1316d59e92fff41a2b78c2df3313bba215f21e0f2c\": rpc error: code = NotFound desc = could not find container \"2f1e5a5370cb01961ac9ef1316d59e92fff41a2b78c2df3313bba215f21e0f2c\": container with ID starting with 2f1e5a5370cb01961ac9ef1316d59e92fff41a2b78c2df3313bba215f21e0f2c not found: ID does not exist"
Sep 16 19:00:35.420958 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.420876    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"]
Sep 16 19:00:35.434285 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:35.434254    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8bed97ae-openshift-default-7d76648678-8c9jm"]
Sep 16 19:00:36.710090 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:36.710056    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" path="/var/lib/kubelet/pods/1aaf86d8-45bd-4ff4-8aa6-391f4980cde0/volumes"
Sep 16 19:00:44.473463 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:44.473426    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj"]
Sep 16 19:00:44.474141 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:44.474120    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" containerName="istio-proxy"
Sep 16 19:00:44.474270 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:44.474144    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" containerName="istio-proxy"
Sep 16 19:00:44.474270 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:44.474198    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" containerName="istio-proxy"
Sep 16 19:00:44.474270 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:44.474208    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" containerName="istio-proxy"
Sep 16 19:00:44.474420 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:44.474326    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" containerName="istio-proxy"
Sep 16 19:00:44.474420 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:44.474339    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aaf86d8-45bd-4ff4-8aa6-391f4980cde0" containerName="istio-proxy"
Sep 16 19:00:44.478799 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:44.478778    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj"
Sep 16 19:00:44.481802 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:44.481780    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nmr62\""
Sep 16 19:00:44.501318 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:44.501290    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj"]
Sep 16 19:00:44.563699 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:44.563664    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e0f7b23-cee9-4d34-a4db-946cd80cc961-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-2efa6f-h76nj\" (UID: \"9e0f7b23-cee9-4d34-a4db-946cd80cc961\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj"
Sep 16 19:00:44.664465 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:44.664422    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e0f7b23-cee9-4d34-a4db-946cd80cc961-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-2efa6f-h76nj\" (UID: \"9e0f7b23-cee9-4d34-a4db-946cd80cc961\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj"
Sep 16 19:00:44.665063 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:44.665035    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e0f7b23-cee9-4d34-a4db-946cd80cc961-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-2efa6f-h76nj\" (UID: \"9e0f7b23-cee9-4d34-a4db-946cd80cc961\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj"
Sep 16 19:00:44.789513 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:44.789422    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj"
Sep 16 19:00:45.077192 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:45.077145    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj"]
Sep 16 19:00:45.078958 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:00:45.078932    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e0f7b23_cee9_4d34_a4db_946cd80cc961.slice/crio-63bcd639a0af27cb80568a7a011b7713eda3268f852e75775fa27f69d26c9b63 WatchSource:0}: Error finding container 63bcd639a0af27cb80568a7a011b7713eda3268f852e75775fa27f69d26c9b63: Status 404 returned error can't find the container with id 63bcd639a0af27cb80568a7a011b7713eda3268f852e75775fa27f69d26c9b63
Sep 16 19:00:45.426343 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:45.426311    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj" event={"ID":"9e0f7b23-cee9-4d34-a4db-946cd80cc961","Type":"ContainerStarted","Data":"ea31bd8a3d7709b29b2fc5701accc5e8d916dad173037fc72a310b537108f20b"}
Sep 16 19:00:45.426531 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:45.426364    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj" event={"ID":"9e0f7b23-cee9-4d34-a4db-946cd80cc961","Type":"ContainerStarted","Data":"63bcd639a0af27cb80568a7a011b7713eda3268f852e75775fa27f69d26c9b63"}
Sep 16 19:00:45.453219 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:45.453147    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj" podStartSLOduration=1.453131327 podStartE2EDuration="1.453131327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:00:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:00:45.452373631 +0000 UTC m=+1273.239937787" watchObservedRunningTime="2026-09-16 19:00:45.453131327 +0000 UTC m=+1273.240695503"
Sep 16 19:00:46.435261 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:46.435230    2578 generic.go:356] "Generic (PLEG): container finished" podID="9e0f7b23-cee9-4d34-a4db-946cd80cc961" containerID="ea31bd8a3d7709b29b2fc5701accc5e8d916dad173037fc72a310b537108f20b" exitCode=0
Sep 16 19:00:46.435650 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:46.435312    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj" event={"ID":"9e0f7b23-cee9-4d34-a4db-946cd80cc961","Type":"ContainerDied","Data":"ea31bd8a3d7709b29b2fc5701accc5e8d916dad173037fc72a310b537108f20b"}
Sep 16 19:00:47.582638 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:47.582613    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj"
Sep 16 19:00:47.695069 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:47.694978    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e0f7b23-cee9-4d34-a4db-946cd80cc961-maas-api-service-ca\") pod \"9e0f7b23-cee9-4d34-a4db-946cd80cc961\" (UID: \"9e0f7b23-cee9-4d34-a4db-946cd80cc961\") "
Sep 16 19:00:47.695390 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:47.695365    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e0f7b23-cee9-4d34-a4db-946cd80cc961-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e0f7b23-cee9-4d34-a4db-946cd80cc961" (UID: "9e0f7b23-cee9-4d34-a4db-946cd80cc961"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:00:47.796957 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:47.796919    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e0f7b23-cee9-4d34-a4db-946cd80cc961-maas-api-service-ca\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:00:48.457545 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:48.457515    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj"
Sep 16 19:00:48.457747 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:48.457550    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj" event={"ID":"9e0f7b23-cee9-4d34-a4db-946cd80cc961","Type":"ContainerDied","Data":"63bcd639a0af27cb80568a7a011b7713eda3268f852e75775fa27f69d26c9b63"}
Sep 16 19:00:48.457747 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:48.457636    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63bcd639a0af27cb80568a7a011b7713eda3268f852e75775fa27f69d26c9b63"
Sep 16 19:00:55.743613 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:55.742811    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2efa6f-29826420-vlv46"]
Sep 16 19:00:55.748751 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:55.748716    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-autores-2efa6f-29826420-vlv46"]
Sep 16 19:00:56.183384 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:56.183342    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj"]
Sep 16 19:00:56.187284 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:56.187256    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-2efa6f-h76nj"]
Sep 16 19:00:56.710900 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:56.710861    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e0f7b23-cee9-4d34-a4db-946cd80cc961" path="/var/lib/kubelet/pods/9e0f7b23-cee9-4d34-a4db-946cd80cc961/volumes"
Sep 16 19:00:56.711298 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:56.711278    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e70a1682-a094-4eb5-9ce9-b84ddbfd1ea5" path="/var/lib/kubelet/pods/e70a1682-a094-4eb5-9ce9-b84ddbfd1ea5/volumes"
Sep 16 19:00:58.532573 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:58.532539    2578 generic.go:356] "Generic (PLEG): container finished" podID="c382fd09-1338-4dcd-a662-36d4483ca83f" containerID="6365410149beccc4dc79ad0455a017d6a23e48cfbef7663ad41c7294b2d1f91b" exitCode=0
Sep 16 19:00:58.533011 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:58.532612    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6" event={"ID":"c382fd09-1338-4dcd-a662-36d4483ca83f","Type":"ContainerDied","Data":"6365410149beccc4dc79ad0455a017d6a23e48cfbef7663ad41c7294b2d1f91b"}
Sep 16 19:00:58.533157 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:58.533137    2578 scope.go:117] "RemoveContainer" containerID="6365410149beccc4dc79ad0455a017d6a23e48cfbef7663ad41c7294b2d1f91b"
Sep 16 19:00:58.534454 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:58.534423    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:00:58.534534 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:58.534493    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:00:58.534534 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:58.534526    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:00:59.543788 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:00:59.543745    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6" event={"ID":"c382fd09-1338-4dcd-a662-36d4483ca83f","Type":"ContainerStarted","Data":"6e176b924e0ad5efcfcfa016e4588ea79061e6a9f5728fc82be26b31419d4e9b"}
Sep 16 19:01:00.145323 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:00.145288    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:01:00.146802 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:00.146771    2578 patch_prober.go:28] interesting pod/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 16 19:01:00.147042 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:00.147014    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6" podUID="c382fd09-1338-4dcd-a662-36d4483ca83f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 16 19:01:01.145938 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:01.145893    2578 patch_prober.go:28] interesting pod/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 16 19:01:01.146458 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:01.145983    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6" podUID="c382fd09-1338-4dcd-a662-36d4483ca83f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 16 19:01:02.145607 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:02.145565    2578 patch_prober.go:28] interesting pod/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 16 19:01:02.145819 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:02.145652    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6" podUID="c382fd09-1338-4dcd-a662-36d4483ca83f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 16 19:01:03.145441 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:03.145401    2578 patch_prober.go:28] interesting pod/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 16 19:01:03.146044 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:03.146007    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6" podUID="c382fd09-1338-4dcd-a662-36d4483ca83f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 16 19:01:03.342808 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:03.342770    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:01:04.146558 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:04.146520    2578 patch_prober.go:28] interesting pod/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 16 19:01:04.147289 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:04.147253    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6" podUID="c382fd09-1338-4dcd-a662-36d4483ca83f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 16 19:01:05.150652 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:05.150598    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:01:05.151864 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:05.151841    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:01:06.977250 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:06.977217    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"]
Sep 16 19:01:06.977725 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:06.977486    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6" podUID="c382fd09-1338-4dcd-a662-36d4483ca83f" containerName="istio-proxy" containerID="cri-o://6e176b924e0ad5efcfcfa016e4588ea79061e6a9f5728fc82be26b31419d4e9b" gracePeriod=30
Sep 16 19:01:12.240643 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.240573    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:01:12.282970 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.282936    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-workload-socket\") pod \"c382fd09-1338-4dcd-a662-36d4483ca83f\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") "
Sep 16 19:01:12.283160 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.282984    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-envoy\") pod \"c382fd09-1338-4dcd-a662-36d4483ca83f\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") "
Sep 16 19:01:12.283160 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283036    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c382fd09-1338-4dcd-a662-36d4483ca83f-istiod-ca-cert\") pod \"c382fd09-1338-4dcd-a662-36d4483ca83f\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") "
Sep 16 19:01:12.283160 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283140    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xfqz\" (UniqueName: \"kubernetes.io/projected/c382fd09-1338-4dcd-a662-36d4483ca83f-kube-api-access-7xfqz\") pod \"c382fd09-1338-4dcd-a662-36d4483ca83f\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") "
Sep 16 19:01:12.283379 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283216    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-data\") pod \"c382fd09-1338-4dcd-a662-36d4483ca83f\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") "
Sep 16 19:01:12.283379 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283253    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-token\") pod \"c382fd09-1338-4dcd-a662-36d4483ca83f\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") "
Sep 16 19:01:12.283379 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283259    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c382fd09-1338-4dcd-a662-36d4483ca83f" (UID: "c382fd09-1338-4dcd-a662-36d4483ca83f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:12.283379 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283282    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-credential-socket\") pod \"c382fd09-1338-4dcd-a662-36d4483ca83f\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") "
Sep 16 19:01:12.283379 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283320    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-workload-certs\") pod \"c382fd09-1338-4dcd-a662-36d4483ca83f\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") "
Sep 16 19:01:12.283652 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283422    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-podinfo\") pod \"c382fd09-1338-4dcd-a662-36d4483ca83f\" (UID: \"c382fd09-1338-4dcd-a662-36d4483ca83f\") "
Sep 16 19:01:12.283652 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283431    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c382fd09-1338-4dcd-a662-36d4483ca83f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c382fd09-1338-4dcd-a662-36d4483ca83f" (UID: "c382fd09-1338-4dcd-a662-36d4483ca83f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:01:12.283652 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283478    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-data" (OuterVolumeSpecName: "istio-data") pod "c382fd09-1338-4dcd-a662-36d4483ca83f" (UID: "c382fd09-1338-4dcd-a662-36d4483ca83f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:12.283826 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283660    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c382fd09-1338-4dcd-a662-36d4483ca83f" (UID: "c382fd09-1338-4dcd-a662-36d4483ca83f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:12.283826 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283709    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c382fd09-1338-4dcd-a662-36d4483ca83f" (UID: "c382fd09-1338-4dcd-a662-36d4483ca83f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:12.283938 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283897    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:12.283938 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283914    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:12.283938 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283923    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:12.283938 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283937    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:12.284147 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.283951    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c382fd09-1338-4dcd-a662-36d4483ca83f-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:12.285803 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.285774    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c382fd09-1338-4dcd-a662-36d4483ca83f" (UID: "c382fd09-1338-4dcd-a662-36d4483ca83f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:12.285940 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.285824    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c382fd09-1338-4dcd-a662-36d4483ca83f" (UID: "c382fd09-1338-4dcd-a662-36d4483ca83f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:01:12.286002 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.285943    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-token" (OuterVolumeSpecName: "istio-token") pod "c382fd09-1338-4dcd-a662-36d4483ca83f" (UID: "c382fd09-1338-4dcd-a662-36d4483ca83f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:01:12.286002 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.285977    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c382fd09-1338-4dcd-a662-36d4483ca83f-kube-api-access-7xfqz" (OuterVolumeSpecName: "kube-api-access-7xfqz") pod "c382fd09-1338-4dcd-a662-36d4483ca83f" (UID: "c382fd09-1338-4dcd-a662-36d4483ca83f"). InnerVolumeSpecName "kube-api-access-7xfqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:01:12.385328 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.385292    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xfqz\" (UniqueName: \"kubernetes.io/projected/c382fd09-1338-4dcd-a662-36d4483ca83f-kube-api-access-7xfqz\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:12.385328 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.385323    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:12.385328 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.385334    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:12.385583 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.385342    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c382fd09-1338-4dcd-a662-36d4483ca83f-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:12.649219 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.649167    2578 generic.go:356] "Generic (PLEG): container finished" podID="c382fd09-1338-4dcd-a662-36d4483ca83f" containerID="6e176b924e0ad5efcfcfa016e4588ea79061e6a9f5728fc82be26b31419d4e9b" exitCode=0
Sep 16 19:01:12.649403 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.649229    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"
Sep 16 19:01:12.649403 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.649246    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6" event={"ID":"c382fd09-1338-4dcd-a662-36d4483ca83f","Type":"ContainerDied","Data":"6e176b924e0ad5efcfcfa016e4588ea79061e6a9f5728fc82be26b31419d4e9b"}
Sep 16 19:01:12.649403 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.649297    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6" event={"ID":"c382fd09-1338-4dcd-a662-36d4483ca83f","Type":"ContainerDied","Data":"73f51d7226eb052296b64deaa4b3a0c56f9fbc24a9a8c497c8b2af2dd8fa3a91"}
Sep 16 19:01:12.649403 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.649321    2578 scope.go:117] "RemoveContainer" containerID="6e176b924e0ad5efcfcfa016e4588ea79061e6a9f5728fc82be26b31419d4e9b"
Sep 16 19:01:12.662518 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.662494    2578 scope.go:117] "RemoveContainer" containerID="6365410149beccc4dc79ad0455a017d6a23e48cfbef7663ad41c7294b2d1f91b"
Sep 16 19:01:12.671970 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.671947    2578 scope.go:117] "RemoveContainer" containerID="6e176b924e0ad5efcfcfa016e4588ea79061e6a9f5728fc82be26b31419d4e9b"
Sep 16 19:01:12.672316 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:01:12.672291    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e176b924e0ad5efcfcfa016e4588ea79061e6a9f5728fc82be26b31419d4e9b\": container with ID starting with 6e176b924e0ad5efcfcfa016e4588ea79061e6a9f5728fc82be26b31419d4e9b not found: ID does not exist" containerID="6e176b924e0ad5efcfcfa016e4588ea79061e6a9f5728fc82be26b31419d4e9b"
Sep 16 19:01:12.672376 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.672326    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e176b924e0ad5efcfcfa016e4588ea79061e6a9f5728fc82be26b31419d4e9b"} err="failed to get container status \"6e176b924e0ad5efcfcfa016e4588ea79061e6a9f5728fc82be26b31419d4e9b\": rpc error: code = NotFound desc = could not find container \"6e176b924e0ad5efcfcfa016e4588ea79061e6a9f5728fc82be26b31419d4e9b\": container with ID starting with 6e176b924e0ad5efcfcfa016e4588ea79061e6a9f5728fc82be26b31419d4e9b not found: ID does not exist"
Sep 16 19:01:12.672376 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.672344    2578 scope.go:117] "RemoveContainer" containerID="6365410149beccc4dc79ad0455a017d6a23e48cfbef7663ad41c7294b2d1f91b"
Sep 16 19:01:12.672600 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:01:12.672579    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6365410149beccc4dc79ad0455a017d6a23e48cfbef7663ad41c7294b2d1f91b\": container with ID starting with 6365410149beccc4dc79ad0455a017d6a23e48cfbef7663ad41c7294b2d1f91b not found: ID does not exist" containerID="6365410149beccc4dc79ad0455a017d6a23e48cfbef7663ad41c7294b2d1f91b"
Sep 16 19:01:12.672693 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.672602    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6365410149beccc4dc79ad0455a017d6a23e48cfbef7663ad41c7294b2d1f91b"} err="failed to get container status \"6365410149beccc4dc79ad0455a017d6a23e48cfbef7663ad41c7294b2d1f91b\": rpc error: code = NotFound desc = could not find container \"6365410149beccc4dc79ad0455a017d6a23e48cfbef7663ad41c7294b2d1f91b\": container with ID starting with 6365410149beccc4dc79ad0455a017d6a23e48cfbef7663ad41c7294b2d1f91b not found: ID does not exist"
Sep 16 19:01:12.678108 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.678081    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"]
Sep 16 19:01:12.680837 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.680816    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-96b42948-openshift-default-c8bd89bcb-ddpp6"]
Sep 16 19:01:12.710066 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:12.710034    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c382fd09-1338-4dcd-a662-36d4483ca83f" path="/var/lib/kubelet/pods/c382fd09-1338-4dcd-a662-36d4483ca83f/volumes"
Sep 16 19:01:40.469841 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:40.469803    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"]
Sep 16 19:01:40.470453 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:40.470035    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98" podUID="ba843615-4a02-4976-ae60-8ceabe7afe30" containerName="istio-proxy" containerID="cri-o://8d5f4cd6314b3f1faf8c18f3f3e6ecffea1f2486d68eb19ed1af43a19192083d" gracePeriod=30
Sep 16 19:01:41.384917 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.384878    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"]
Sep 16 19:01:41.385341 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.385327    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e0f7b23-cee9-4d34-a4db-946cd80cc961" containerName="revoke-keys"
Sep 16 19:01:41.385407 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.385343    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e0f7b23-cee9-4d34-a4db-946cd80cc961" containerName="revoke-keys"
Sep 16 19:01:41.385407 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.385354    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c382fd09-1338-4dcd-a662-36d4483ca83f" containerName="istio-proxy"
Sep 16 19:01:41.385407 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.385366    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c382fd09-1338-4dcd-a662-36d4483ca83f" containerName="istio-proxy"
Sep 16 19:01:41.385407 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.385382    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c382fd09-1338-4dcd-a662-36d4483ca83f" containerName="istio-proxy"
Sep 16 19:01:41.385407 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.385388    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c382fd09-1338-4dcd-a662-36d4483ca83f" containerName="istio-proxy"
Sep 16 19:01:41.385566 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.385458    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c382fd09-1338-4dcd-a662-36d4483ca83f" containerName="istio-proxy"
Sep 16 19:01:41.385566 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.385464    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e0f7b23-cee9-4d34-a4db-946cd80cc961" containerName="revoke-keys"
Sep 16 19:01:41.390113 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.390080    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.394335 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.394311    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-9d2450-openshift-default-dockercfg-7j7cd\""
Sep 16 19:01:41.406663 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.406631    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"]
Sep 16 19:01:41.493208 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.493153    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2582bb11-32ae-486b-bae6-65b64397fba6-istiod-ca-cert\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.493813 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.493790    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvlk4\" (UniqueName: \"kubernetes.io/projected/2582bb11-32ae-486b-bae6-65b64397fba6-kube-api-access-kvlk4\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.493931 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.493916    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2582bb11-32ae-486b-bae6-65b64397fba6-istio-podinfo\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.494071 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.494059    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2582bb11-32ae-486b-bae6-65b64397fba6-istio-token\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.494264 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.494245    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-workload-socket\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.494434 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.494409    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-istio-envoy\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.494601 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.494587    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-credential-socket\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.494729 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.494715    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-workload-certs\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.494825 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.494813    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-istio-data\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.596108 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.596072    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvlk4\" (UniqueName: \"kubernetes.io/projected/2582bb11-32ae-486b-bae6-65b64397fba6-kube-api-access-kvlk4\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.596108 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.596113    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2582bb11-32ae-486b-bae6-65b64397fba6-istio-podinfo\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.596419 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.596150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2582bb11-32ae-486b-bae6-65b64397fba6-istio-token\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.596419 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.596220    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-workload-socket\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.596419 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.596238    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-istio-envoy\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.596419 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.596275    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-credential-socket\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.596419 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.596321    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-workload-certs\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.596419 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.596345    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-istio-data\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.596742 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.596436    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2582bb11-32ae-486b-bae6-65b64397fba6-istiod-ca-cert\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.596795 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.596744    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-workload-socket\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.596839 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.596799    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-workload-certs\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.596839 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.596825    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-istio-data\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.597118 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.597097    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-credential-socket\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.597226 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.597207    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2582bb11-32ae-486b-bae6-65b64397fba6-istiod-ca-cert\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.599004 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.598982    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2582bb11-32ae-486b-bae6-65b64397fba6-istio-podinfo\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.599097 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.599066    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-istio-envoy\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.611960 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.611932    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2582bb11-32ae-486b-bae6-65b64397fba6-istio-token\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.612688 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.612671    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvlk4\" (UniqueName: \"kubernetes.io/projected/2582bb11-32ae-486b-bae6-65b64397fba6-kube-api-access-kvlk4\") pod \"e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") " pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.700946 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.700853    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:41.997685 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:41.997653    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"]
Sep 16 19:01:41.999110 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:01:41.999082    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2582bb11_32ae_486b_bae6_65b64397fba6.slice/crio-23769da7219e41641030759fa40d746abdf9b8219c145b8bdc8e7e4b48a70ea4 WatchSource:0}: Error finding container 23769da7219e41641030759fa40d746abdf9b8219c145b8bdc8e7e4b48a70ea4: Status 404 returned error can't find the container with id 23769da7219e41641030759fa40d746abdf9b8219c145b8bdc8e7e4b48a70ea4
Sep 16 19:01:42.001373 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:42.001335    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:01:42.001451 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:42.001406    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:01:42.001451 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:42.001434    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:01:42.845240 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:42.845196    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d" event={"ID":"2582bb11-32ae-486b-bae6-65b64397fba6","Type":"ContainerStarted","Data":"ca86029fd67c378a226b7c47ecc8cbbb63c51f1caa43ef76b8f321e00ef8b9a2"}
Sep 16 19:01:42.845652 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:42.845267    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d" event={"ID":"2582bb11-32ae-486b-bae6-65b64397fba6","Type":"ContainerStarted","Data":"23769da7219e41641030759fa40d746abdf9b8219c145b8bdc8e7e4b48a70ea4"}
Sep 16 19:01:42.876275 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:42.876224    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d" podStartSLOduration=1.876207626 podStartE2EDuration="1.876207626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:01:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:01:42.873751008 +0000 UTC m=+1330.661315188" watchObservedRunningTime="2026-09-16 19:01:42.876207626 +0000 UTC m=+1330.663771781"
Sep 16 19:01:43.701522 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:43.701486    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:43.706803 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:43.706773    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:43.851809 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:43.851770    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:43.852914 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:43.852887    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:01:45.719874 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.719850    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 19:01:45.742327 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.742257    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-envoy\") pod \"ba843615-4a02-4976-ae60-8ceabe7afe30\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") "
Sep 16 19:01:45.742327 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.742300    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba843615-4a02-4976-ae60-8ceabe7afe30-istiod-ca-cert\") pod \"ba843615-4a02-4976-ae60-8ceabe7afe30\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") "
Sep 16 19:01:45.742518 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.742330    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9r7w5\" (UniqueName: \"kubernetes.io/projected/ba843615-4a02-4976-ae60-8ceabe7afe30-kube-api-access-9r7w5\") pod \"ba843615-4a02-4976-ae60-8ceabe7afe30\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") "
Sep 16 19:01:45.742518 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.742433    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-workload-certs\") pod \"ba843615-4a02-4976-ae60-8ceabe7afe30\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") "
Sep 16 19:01:45.742518 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.742478    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-token\") pod \"ba843615-4a02-4976-ae60-8ceabe7afe30\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") "
Sep 16 19:01:45.742638 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.742556    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-workload-socket\") pod \"ba843615-4a02-4976-ae60-8ceabe7afe30\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") "
Sep 16 19:01:45.742638 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.742592    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-data\") pod \"ba843615-4a02-4976-ae60-8ceabe7afe30\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") "
Sep 16 19:01:45.742734 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.742674    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ba843615-4a02-4976-ae60-8ceabe7afe30" (UID: "ba843615-4a02-4976-ae60-8ceabe7afe30"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:45.742804 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.742671    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba843615-4a02-4976-ae60-8ceabe7afe30-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ba843615-4a02-4976-ae60-8ceabe7afe30" (UID: "ba843615-4a02-4976-ae60-8ceabe7afe30"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:01:45.742804 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.742715    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-podinfo\") pod \"ba843615-4a02-4976-ae60-8ceabe7afe30\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") "
Sep 16 19:01:45.742903 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.742807    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-credential-socket\") pod \"ba843615-4a02-4976-ae60-8ceabe7afe30\" (UID: \"ba843615-4a02-4976-ae60-8ceabe7afe30\") "
Sep 16 19:01:45.742959 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.742898    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-data" (OuterVolumeSpecName: "istio-data") pod "ba843615-4a02-4976-ae60-8ceabe7afe30" (UID: "ba843615-4a02-4976-ae60-8ceabe7afe30"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:45.743041 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.743019    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ba843615-4a02-4976-ae60-8ceabe7afe30" (UID: "ba843615-4a02-4976-ae60-8ceabe7afe30"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:45.743188 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.743150    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ba843615-4a02-4976-ae60-8ceabe7afe30" (UID: "ba843615-4a02-4976-ae60-8ceabe7afe30"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:45.743590 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.743566    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:45.743681 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.743595    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:45.743681 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.743611    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:45.743681 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.743627    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:45.743681 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.743641    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba843615-4a02-4976-ae60-8ceabe7afe30-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:45.744825 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.744784    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba843615-4a02-4976-ae60-8ceabe7afe30-kube-api-access-9r7w5" (OuterVolumeSpecName: "kube-api-access-9r7w5") pod "ba843615-4a02-4976-ae60-8ceabe7afe30" (UID: "ba843615-4a02-4976-ae60-8ceabe7afe30"). InnerVolumeSpecName "kube-api-access-9r7w5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:01:45.744943 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.744906    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ba843615-4a02-4976-ae60-8ceabe7afe30" (UID: "ba843615-4a02-4976-ae60-8ceabe7afe30"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:01:45.745079 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.745054    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ba843615-4a02-4976-ae60-8ceabe7afe30" (UID: "ba843615-4a02-4976-ae60-8ceabe7afe30"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:45.745274 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.745252    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-token" (OuterVolumeSpecName: "istio-token") pod "ba843615-4a02-4976-ae60-8ceabe7afe30" (UID: "ba843615-4a02-4976-ae60-8ceabe7afe30"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:01:45.844766 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.844729    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:45.844766 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.844762    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:45.844766 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.844770    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba843615-4a02-4976-ae60-8ceabe7afe30-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:45.845030 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.844779    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r7w5\" (UniqueName: \"kubernetes.io/projected/ba843615-4a02-4976-ae60-8ceabe7afe30-kube-api-access-9r7w5\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:01:45.867113 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.867080    2578 generic.go:356] "Generic (PLEG): container finished" podID="ba843615-4a02-4976-ae60-8ceabe7afe30" containerID="8d5f4cd6314b3f1faf8c18f3f3e6ecffea1f2486d68eb19ed1af43a19192083d" exitCode=0
Sep 16 19:01:45.867352 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.867138    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"
Sep 16 19:01:45.867352 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.867164    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98" event={"ID":"ba843615-4a02-4976-ae60-8ceabe7afe30","Type":"ContainerDied","Data":"8d5f4cd6314b3f1faf8c18f3f3e6ecffea1f2486d68eb19ed1af43a19192083d"}
Sep 16 19:01:45.867352 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.867257    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98" event={"ID":"ba843615-4a02-4976-ae60-8ceabe7afe30","Type":"ContainerDied","Data":"1682b1593725965d53449ffb9495c96a46501faa0fbaa58a907b085a6835059e"}
Sep 16 19:01:45.867352 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.867280    2578 scope.go:117] "RemoveContainer" containerID="8d5f4cd6314b3f1faf8c18f3f3e6ecffea1f2486d68eb19ed1af43a19192083d"
Sep 16 19:01:45.877624 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.877606    2578 scope.go:117] "RemoveContainer" containerID="6d14991454de6ad1ece791a876e46fec3876cca44e77900a37c89471a2544b73"
Sep 16 19:01:45.886422 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.886405    2578 scope.go:117] "RemoveContainer" containerID="8d5f4cd6314b3f1faf8c18f3f3e6ecffea1f2486d68eb19ed1af43a19192083d"
Sep 16 19:01:45.886686 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:01:45.886669    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d5f4cd6314b3f1faf8c18f3f3e6ecffea1f2486d68eb19ed1af43a19192083d\": container with ID starting with 8d5f4cd6314b3f1faf8c18f3f3e6ecffea1f2486d68eb19ed1af43a19192083d not found: ID does not exist" containerID="8d5f4cd6314b3f1faf8c18f3f3e6ecffea1f2486d68eb19ed1af43a19192083d"
Sep 16 19:01:45.886731 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.886694    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d5f4cd6314b3f1faf8c18f3f3e6ecffea1f2486d68eb19ed1af43a19192083d"} err="failed to get container status \"8d5f4cd6314b3f1faf8c18f3f3e6ecffea1f2486d68eb19ed1af43a19192083d\": rpc error: code = NotFound desc = could not find container \"8d5f4cd6314b3f1faf8c18f3f3e6ecffea1f2486d68eb19ed1af43a19192083d\": container with ID starting with 8d5f4cd6314b3f1faf8c18f3f3e6ecffea1f2486d68eb19ed1af43a19192083d not found: ID does not exist"
Sep 16 19:01:45.886731 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.886710    2578 scope.go:117] "RemoveContainer" containerID="6d14991454de6ad1ece791a876e46fec3876cca44e77900a37c89471a2544b73"
Sep 16 19:01:45.886933 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:01:45.886917    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d14991454de6ad1ece791a876e46fec3876cca44e77900a37c89471a2544b73\": container with ID starting with 6d14991454de6ad1ece791a876e46fec3876cca44e77900a37c89471a2544b73 not found: ID does not exist" containerID="6d14991454de6ad1ece791a876e46fec3876cca44e77900a37c89471a2544b73"
Sep 16 19:01:45.886969 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.886938    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d14991454de6ad1ece791a876e46fec3876cca44e77900a37c89471a2544b73"} err="failed to get container status \"6d14991454de6ad1ece791a876e46fec3876cca44e77900a37c89471a2544b73\": rpc error: code = NotFound desc = could not find container \"6d14991454de6ad1ece791a876e46fec3876cca44e77900a37c89471a2544b73\": container with ID starting with 6d14991454de6ad1ece791a876e46fec3876cca44e77900a37c89471a2544b73 not found: ID does not exist"
Sep 16 19:01:45.895450 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.895426    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"]
Sep 16 19:01:45.899742 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:45.899718    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-2efa6f-openshift-default-7b8cc466fd-gwz98"]
Sep 16 19:01:46.713533 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:01:46.713497    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba843615-4a02-4976-ae60-8ceabe7afe30" path="/var/lib/kubelet/pods/ba843615-4a02-4976-ae60-8ceabe7afe30/volumes"
Sep 16 19:02:05.384526 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.384437    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"]
Sep 16 19:02:05.384903 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.384870    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba843615-4a02-4976-ae60-8ceabe7afe30" containerName="istio-proxy"
Sep 16 19:02:05.384903 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.384881    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba843615-4a02-4976-ae60-8ceabe7afe30" containerName="istio-proxy"
Sep 16 19:02:05.384903 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.384890    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba843615-4a02-4976-ae60-8ceabe7afe30" containerName="istio-proxy"
Sep 16 19:02:05.384903 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.384897    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba843615-4a02-4976-ae60-8ceabe7afe30" containerName="istio-proxy"
Sep 16 19:02:05.385061 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.384993    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c382fd09-1338-4dcd-a662-36d4483ca83f" containerName="istio-proxy"
Sep 16 19:02:05.385061 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.385009    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba843615-4a02-4976-ae60-8ceabe7afe30" containerName="istio-proxy"
Sep 16 19:02:05.385061 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.385017    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba843615-4a02-4976-ae60-8ceabe7afe30" containerName="istio-proxy"
Sep 16 19:02:05.388259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.388239    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.391936 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.391909    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8a45a9\"/\"kube-root-ca.crt\""
Sep 16 19:02:05.392083 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.391909    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8a45a9\"/\"default-dockercfg-5m7vw\""
Sep 16 19:02:05.392083 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.391916    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8a45a9\"/\"test-model-8a45a9-kserve-self-signed-certs\""
Sep 16 19:02:05.392083 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.391971    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8a45a9\"/\"openshift-service-ca.crt\""
Sep 16 19:02:05.401913 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.401887    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"]
Sep 16 19:02:05.556742 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.556702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-home\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.556905 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.556750    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcff8da3-8c6c-461f-a622-1eb1f57c091f-tls-certs\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.556905 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.556840    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-model-cache\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.556905 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.556901    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-dshm\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.557018 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.556930    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ldrn\" (UniqueName: \"kubernetes.io/projected/bcff8da3-8c6c-461f-a622-1eb1f57c091f-kube-api-access-8ldrn\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.557018 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.556956    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-tmp-dir\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.658022 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.657930    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-dshm\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.658022 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.657969    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ldrn\" (UniqueName: \"kubernetes.io/projected/bcff8da3-8c6c-461f-a622-1eb1f57c091f-kube-api-access-8ldrn\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.658022 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.657999    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-tmp-dir\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.658358 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.658091    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-home\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.658358 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.658125    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcff8da3-8c6c-461f-a622-1eb1f57c091f-tls-certs\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.658358 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.658151    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-model-cache\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.658564 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.658537    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-tmp-dir\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.658724 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.658702    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-home\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.658812 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.658754    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-model-cache\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.660358 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.660329    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-dshm\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.660658 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.660638    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcff8da3-8c6c-461f-a622-1eb1f57c091f-tls-certs\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.673464 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.673437    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ldrn\" (UniqueName: \"kubernetes.io/projected/bcff8da3-8c6c-461f-a622-1eb1f57c091f-kube-api-access-8ldrn\") pod \"test-model-8a45a9-kserve-679db5c4b9-q28rw\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") " pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.700518 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.700484    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:05.991355 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:05.991323    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"]
Sep 16 19:02:05.992387 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:02:05.992358    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbcff8da3_8c6c_461f_a622_1eb1f57c091f.slice/crio-5da96c5be3385cebc112b23fca23a2857fe79330359b2354804e7873324079eb WatchSource:0}: Error finding container 5da96c5be3385cebc112b23fca23a2857fe79330359b2354804e7873324079eb: Status 404 returned error can't find the container with id 5da96c5be3385cebc112b23fca23a2857fe79330359b2354804e7873324079eb
Sep 16 19:02:06.011257 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:06.011219    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw" event={"ID":"bcff8da3-8c6c-461f-a622-1eb1f57c091f","Type":"ContainerStarted","Data":"5da96c5be3385cebc112b23fca23a2857fe79330359b2354804e7873324079eb"}
Sep 16 19:02:07.019550 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:07.019501    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw" event={"ID":"bcff8da3-8c6c-461f-a622-1eb1f57c091f","Type":"ContainerStarted","Data":"6b989acd46c5cec0b2e684ab292040bd52135fb59032f08f85bff1f4dbcd1ac1"}
Sep 16 19:02:09.033758 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:09.033729    2578 generic.go:356] "Generic (PLEG): container finished" podID="2582bb11-32ae-486b-bae6-65b64397fba6" containerID="ca86029fd67c378a226b7c47ecc8cbbb63c51f1caa43ef76b8f321e00ef8b9a2" exitCode=0
Sep 16 19:02:09.034334 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:09.033806    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d" event={"ID":"2582bb11-32ae-486b-bae6-65b64397fba6","Type":"ContainerDied","Data":"ca86029fd67c378a226b7c47ecc8cbbb63c51f1caa43ef76b8f321e00ef8b9a2"}
Sep 16 19:02:09.034409 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:09.034387    2578 scope.go:117] "RemoveContainer" containerID="ca86029fd67c378a226b7c47ecc8cbbb63c51f1caa43ef76b8f321e00ef8b9a2"
Sep 16 19:02:09.035634 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:09.035568    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:02:09.035740 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:09.035687    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:02:09.035740 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:09.035724    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:02:09.058326 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:09.058229    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw" podStartSLOduration=4.058208958 podStartE2EDuration="4.058208958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:02:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:02:07.044971254 +0000 UTC m=+1354.832535428" watchObservedRunningTime="2026-09-16 19:02:09.058208958 +0000 UTC m=+1356.845773110"
Sep 16 19:02:10.044576 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:10.044540    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d" event={"ID":"2582bb11-32ae-486b-bae6-65b64397fba6","Type":"ContainerStarted","Data":"3ee8d6366f5a9332050ec5e0f0a6c3ebfbd74b8665d81ec34203465d7e82cf52"}
Sep 16 19:02:10.701224 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:10.701155    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:02:10.702436 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:10.702413    2578 patch_prober.go:28] interesting pod/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 16 19:02:10.702529 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:10.702463    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d" podUID="2582bb11-32ae-486b-bae6-65b64397fba6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 16 19:02:11.701833 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:11.701798    2578 patch_prober.go:28] interesting pod/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 16 19:02:11.702249 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:11.701852    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d" podUID="2582bb11-32ae-486b-bae6-65b64397fba6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 16 19:02:12.701277 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:12.701238    2578 patch_prober.go:28] interesting pod/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 16 19:02:12.701436 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:12.701299    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d" podUID="2582bb11-32ae-486b-bae6-65b64397fba6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 16 19:02:13.701223 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:13.701187    2578 patch_prober.go:28] interesting pod/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 16 19:02:13.701695 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:13.701260    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d" podUID="2582bb11-32ae-486b-bae6-65b64397fba6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 16 19:02:13.852215 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:13.852153    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:02:14.701733 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:14.701696    2578 patch_prober.go:28] interesting pod/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 16 19:02:14.702092 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:14.701758    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d" podUID="2582bb11-32ae-486b-bae6-65b64397fba6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 16 19:02:15.701149 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:15.701109    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:15.701461 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:15.701163    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:15.701461 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:15.701420    2578 patch_prober.go:28] interesting pod/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 16 19:02:15.701586 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:15.701484    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d" podUID="2582bb11-32ae-486b-bae6-65b64397fba6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 16 19:02:15.714520 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:15.714494    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:16.106842 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:16.106819    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:02:16.709783 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:16.709755    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:02:16.710878 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:16.710859    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:02:50.693613 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.693579    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"]
Sep 16 19:02:50.698701 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.698680    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.702259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.702235    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-050f3bde-openshift-default-dockercfg-95zr9\""
Sep 16 19:02:50.712852 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.712822    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"]
Sep 16 19:02:50.752774 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.752743    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istiod-ca-cert\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.752972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.752779    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-workload-socket\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.752972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.752803    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-envoy\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.752972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.752905    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-podinfo\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.753157 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.753011    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-credential-socket\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.753157 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.753068    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-workload-certs\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.753157 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.753095    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlwxx\" (UniqueName: \"kubernetes.io/projected/e5a645e4-76dc-4b52-8ac0-02a25b06af42-kube-api-access-nlwxx\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.753157 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.753123    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-token\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.753157 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.753153    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-data\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.854657 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.854609    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-credential-socket\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.854855 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.854707    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-workload-certs\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.855292 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.855263    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-credential-socket\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.855432 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.855346    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlwxx\" (UniqueName: \"kubernetes.io/projected/e5a645e4-76dc-4b52-8ac0-02a25b06af42-kube-api-access-nlwxx\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.855432 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.855392    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-token\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.855432 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.855422    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-data\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.855587 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.855529    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istiod-ca-cert\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.855587 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.855572    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-workload-socket\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.855689 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.855610    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-envoy\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.855689 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.855651    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-podinfo\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.856590 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.856562    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-data\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.856837 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.856816    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-workload-certs\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.857740 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.857711    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-workload-socket\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.858377 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.858351    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istiod-ca-cert\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.863296 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.863205    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-envoy\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.864083 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.864054    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-podinfo\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.868430 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.868408    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-token\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:50.868626 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:50.868598    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlwxx\" (UniqueName: \"kubernetes.io/projected/e5a645e4-76dc-4b52-8ac0-02a25b06af42-kube-api-access-nlwxx\") pod \"e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") " pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:51.013442 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:51.013336    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:51.385778 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:51.385715    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"]
Sep 16 19:02:51.388361 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:02:51.388333    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5a645e4_76dc_4b52_8ac0_02a25b06af42.slice/crio-655d648a3092ae1c534311393948a4c21a0858fb29eb9f16b096fbdbcc4fd68e WatchSource:0}: Error finding container 655d648a3092ae1c534311393948a4c21a0858fb29eb9f16b096fbdbcc4fd68e: Status 404 returned error can't find the container with id 655d648a3092ae1c534311393948a4c21a0858fb29eb9f16b096fbdbcc4fd68e
Sep 16 19:02:51.390770 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:51.390725    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:02:51.390865 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:51.390847    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:02:51.390909 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:51.390894    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:02:52.348329 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:52.348293    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv" event={"ID":"e5a645e4-76dc-4b52-8ac0-02a25b06af42","Type":"ContainerStarted","Data":"5a4475fb2b6cd7bf5291101b215c5c6340e33faf39b4544fabba250563b0c7d0"}
Sep 16 19:02:52.348716 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:52.348344    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv" event={"ID":"e5a645e4-76dc-4b52-8ac0-02a25b06af42","Type":"ContainerStarted","Data":"655d648a3092ae1c534311393948a4c21a0858fb29eb9f16b096fbdbcc4fd68e"}
Sep 16 19:02:52.375716 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:52.375665    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv" podStartSLOduration=2.375647306 podStartE2EDuration="2.375647306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:02:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:02:52.371408307 +0000 UTC m=+1400.158972463" watchObservedRunningTime="2026-09-16 19:02:52.375647306 +0000 UTC m=+1400.163211460"
Sep 16 19:02:53.013927 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:53.013887    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:53.018833 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:53.018809    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:53.356357 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:53.356331    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:02:53.357243 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:02:53.357225    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:03:29.613157 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:29.613121    2578 generic.go:356] "Generic (PLEG): container finished" podID="e5a645e4-76dc-4b52-8ac0-02a25b06af42" containerID="5a4475fb2b6cd7bf5291101b215c5c6340e33faf39b4544fabba250563b0c7d0" exitCode=0
Sep 16 19:03:29.613600 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:29.613192    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv" event={"ID":"e5a645e4-76dc-4b52-8ac0-02a25b06af42","Type":"ContainerDied","Data":"5a4475fb2b6cd7bf5291101b215c5c6340e33faf39b4544fabba250563b0c7d0"}
Sep 16 19:03:29.613648 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:29.613631    2578 scope.go:117] "RemoveContainer" containerID="5a4475fb2b6cd7bf5291101b215c5c6340e33faf39b4544fabba250563b0c7d0"
Sep 16 19:03:29.614903 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:29.614869    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:03:29.614991 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:29.614940    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:03:29.614991 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:29.614975    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:03:30.621867 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:30.621835    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv" event={"ID":"e5a645e4-76dc-4b52-8ac0-02a25b06af42","Type":"ContainerStarted","Data":"e17180c9f9ce5efa2045df448bf4c1aa9e01544211ef74c31079ed6ffad0729d"}
Sep 16 19:03:31.014630 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:31.014537    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:03:31.014769 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:31.014742    2578 patch_prober.go:28] interesting pod/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 16 19:03:31.014812 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:31.014787    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv" podUID="e5a645e4-76dc-4b52-8ac0-02a25b06af42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 16 19:03:32.014783 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:32.014744    2578 patch_prober.go:28] interesting pod/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 16 19:03:32.015208 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:32.014800    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv" podUID="e5a645e4-76dc-4b52-8ac0-02a25b06af42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 16 19:03:33.014135 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:33.014094    2578 patch_prober.go:28] interesting pod/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 16 19:03:33.014368 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:33.014161    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv" podUID="e5a645e4-76dc-4b52-8ac0-02a25b06af42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 16 19:03:34.013968 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:34.013921    2578 patch_prober.go:28] interesting pod/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 16 19:03:34.014488 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:34.013987    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv" podUID="e5a645e4-76dc-4b52-8ac0-02a25b06af42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 16 19:03:35.013993 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:35.013956    2578 patch_prober.go:28] interesting pod/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 16 19:03:35.014385 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:35.014023    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv" podUID="e5a645e4-76dc-4b52-8ac0-02a25b06af42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 16 19:03:36.019543 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:36.019510    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:03:36.019972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:36.019811    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:03:36.020507 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:36.020489    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:03:36.632377 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:36.632341    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf"]
Sep 16 19:03:36.635138 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:36.635120    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf"
Sep 16 19:03:36.637559 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:36.637537    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nmr62\""
Sep 16 19:03:36.649419 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:36.649383    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf"]
Sep 16 19:03:36.758874 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:36.758832    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf\" (UID: \"56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf"
Sep 16 19:03:36.859628 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:36.859589    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf\" (UID: \"56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf"
Sep 16 19:03:36.860229 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:36.860207    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf\" (UID: \"56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf"
Sep 16 19:03:36.947660 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:36.947566    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf"
Sep 16 19:03:37.225129 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:37.225105    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf"]
Sep 16 19:03:37.226369 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:03:37.226344    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56b76fb3_fb63_45b6_ad4e_fbdd47ae1fe2.slice/crio-ec43e8f16d85bff6e7ce3080bbc0d3fe2104ab03499e8f13964de41320c1bb6a WatchSource:0}: Error finding container ec43e8f16d85bff6e7ce3080bbc0d3fe2104ab03499e8f13964de41320c1bb6a: Status 404 returned error can't find the container with id ec43e8f16d85bff6e7ce3080bbc0d3fe2104ab03499e8f13964de41320c1bb6a
Sep 16 19:03:37.678261 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:37.678210    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf" event={"ID":"56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2","Type":"ContainerStarted","Data":"49c98e611129c28b5cab23c27b5cd22f2117195772a31c71b0110c000b3e082f"}
Sep 16 19:03:37.678452 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:37.678276    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf" event={"ID":"56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2","Type":"ContainerStarted","Data":"ec43e8f16d85bff6e7ce3080bbc0d3fe2104ab03499e8f13964de41320c1bb6a"}
Sep 16 19:03:37.700146 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:37.700095    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf" podStartSLOduration=1.700076473 podStartE2EDuration="1.700076473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:03:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:03:37.69989272 +0000 UTC m=+1445.487456893" watchObservedRunningTime="2026-09-16 19:03:37.700076473 +0000 UTC m=+1445.487640630"
Sep 16 19:03:38.690566 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:38.690535    2578 generic.go:356] "Generic (PLEG): container finished" podID="56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2" containerID="49c98e611129c28b5cab23c27b5cd22f2117195772a31c71b0110c000b3e082f" exitCode=0
Sep 16 19:03:38.690974 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:38.690617    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf" event={"ID":"56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2","Type":"ContainerDied","Data":"49c98e611129c28b5cab23c27b5cd22f2117195772a31c71b0110c000b3e082f"}
Sep 16 19:03:39.837154 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:39.837130    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf"
Sep 16 19:03:39.996642 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:39.996547    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2-maas-api-service-ca\") pod \"56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2\" (UID: \"56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2\") "
Sep 16 19:03:39.996927 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:39.996902    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2" (UID: "56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:03:40.097691 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:40.097650    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2-maas-api-service-ca\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:03:40.705192 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:40.705147    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf"
Sep 16 19:03:40.710279 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:40.710244    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf" event={"ID":"56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2","Type":"ContainerDied","Data":"ec43e8f16d85bff6e7ce3080bbc0d3fe2104ab03499e8f13964de41320c1bb6a"}
Sep 16 19:03:40.710462 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:40.710293    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec43e8f16d85bff6e7ce3080bbc0d3fe2104ab03499e8f13964de41320c1bb6a"
Sep 16 19:03:47.869413 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:47.869375    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf"]
Sep 16 19:03:47.872305 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:47.872275    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-050f3bde-6xkxf"]
Sep 16 19:03:48.709367 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:48.709333    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2" path="/var/lib/kubelet/pods/56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2/volumes"
Sep 16 19:03:55.873803 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:55.873771    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"]
Sep 16 19:03:55.874275 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:03:55.873986    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv" podUID="e5a645e4-76dc-4b52-8ac0-02a25b06af42" containerName="istio-proxy" containerID="cri-o://e17180c9f9ce5efa2045df448bf4c1aa9e01544211ef74c31079ed6ffad0729d" gracePeriod=30
Sep 16 19:04:01.133450 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.133424    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:04:01.228892 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.228803    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-workload-certs\") pod \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") "
Sep 16 19:04:01.228892 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.228848    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-data\") pod \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") "
Sep 16 19:04:01.228892 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.228879    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-workload-socket\") pod \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") "
Sep 16 19:04:01.229131 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.228917    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-podinfo\") pod \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") "
Sep 16 19:04:01.229131 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.228962    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istiod-ca-cert\") pod \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") "
Sep 16 19:04:01.229131 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.229006    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-token\") pod \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") "
Sep 16 19:04:01.229131 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.229079    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlwxx\" (UniqueName: \"kubernetes.io/projected/e5a645e4-76dc-4b52-8ac0-02a25b06af42-kube-api-access-nlwxx\") pod \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") "
Sep 16 19:04:01.229131 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.229108    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-envoy\") pod \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") "
Sep 16 19:04:01.229449 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.229139    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-credential-socket\") pod \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\" (UID: \"e5a645e4-76dc-4b52-8ac0-02a25b06af42\") "
Sep 16 19:04:01.229449 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.229130    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-data" (OuterVolumeSpecName: "istio-data") pod "e5a645e4-76dc-4b52-8ac0-02a25b06af42" (UID: "e5a645e4-76dc-4b52-8ac0-02a25b06af42"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:04:01.229449 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.229154    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e5a645e4-76dc-4b52-8ac0-02a25b06af42" (UID: "e5a645e4-76dc-4b52-8ac0-02a25b06af42"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:04:01.229449 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.229285    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e5a645e4-76dc-4b52-8ac0-02a25b06af42" (UID: "e5a645e4-76dc-4b52-8ac0-02a25b06af42"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:04:01.229449 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.229424    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e5a645e4-76dc-4b52-8ac0-02a25b06af42" (UID: "e5a645e4-76dc-4b52-8ac0-02a25b06af42"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:04:01.229732 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.229659    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e5a645e4-76dc-4b52-8ac0-02a25b06af42" (UID: "e5a645e4-76dc-4b52-8ac0-02a25b06af42"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:04:01.229732 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.229691    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:04:01.229732 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.229712    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:04:01.229732 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.229725    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:04:01.229958 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.229738    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:04:01.231348 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.231323    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e5a645e4-76dc-4b52-8ac0-02a25b06af42" (UID: "e5a645e4-76dc-4b52-8ac0-02a25b06af42"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:04:01.231348 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.231336    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5a645e4-76dc-4b52-8ac0-02a25b06af42-kube-api-access-nlwxx" (OuterVolumeSpecName: "kube-api-access-nlwxx") pod "e5a645e4-76dc-4b52-8ac0-02a25b06af42" (UID: "e5a645e4-76dc-4b52-8ac0-02a25b06af42"). InnerVolumeSpecName "kube-api-access-nlwxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:04:01.231601 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.231577    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-token" (OuterVolumeSpecName: "istio-token") pod "e5a645e4-76dc-4b52-8ac0-02a25b06af42" (UID: "e5a645e4-76dc-4b52-8ac0-02a25b06af42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:04:01.231679 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.231595    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e5a645e4-76dc-4b52-8ac0-02a25b06af42" (UID: "e5a645e4-76dc-4b52-8ac0-02a25b06af42"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:04:01.330656 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.330618    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:04:01.330656 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.330649    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:04:01.330656 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.330661    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:04:01.330916 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.330674    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlwxx\" (UniqueName: \"kubernetes.io/projected/e5a645e4-76dc-4b52-8ac0-02a25b06af42-kube-api-access-nlwxx\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:04:01.330916 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.330686    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e5a645e4-76dc-4b52-8ac0-02a25b06af42-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:04:01.865299 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.865264    2578 generic.go:356] "Generic (PLEG): container finished" podID="e5a645e4-76dc-4b52-8ac0-02a25b06af42" containerID="e17180c9f9ce5efa2045df448bf4c1aa9e01544211ef74c31079ed6ffad0729d" exitCode=0
Sep 16 19:04:01.865481 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.865312    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"
Sep 16 19:04:01.865481 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.865342    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv" event={"ID":"e5a645e4-76dc-4b52-8ac0-02a25b06af42","Type":"ContainerDied","Data":"e17180c9f9ce5efa2045df448bf4c1aa9e01544211ef74c31079ed6ffad0729d"}
Sep 16 19:04:01.865481 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.865421    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv" event={"ID":"e5a645e4-76dc-4b52-8ac0-02a25b06af42","Type":"ContainerDied","Data":"655d648a3092ae1c534311393948a4c21a0858fb29eb9f16b096fbdbcc4fd68e"}
Sep 16 19:04:01.865481 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.865445    2578 scope.go:117] "RemoveContainer" containerID="e17180c9f9ce5efa2045df448bf4c1aa9e01544211ef74c31079ed6ffad0729d"
Sep 16 19:04:01.876672 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.876653    2578 scope.go:117] "RemoveContainer" containerID="5a4475fb2b6cd7bf5291101b215c5c6340e33faf39b4544fabba250563b0c7d0"
Sep 16 19:04:01.886000 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.885978    2578 scope.go:117] "RemoveContainer" containerID="e17180c9f9ce5efa2045df448bf4c1aa9e01544211ef74c31079ed6ffad0729d"
Sep 16 19:04:01.886473 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:04:01.886436    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e17180c9f9ce5efa2045df448bf4c1aa9e01544211ef74c31079ed6ffad0729d\": container with ID starting with e17180c9f9ce5efa2045df448bf4c1aa9e01544211ef74c31079ed6ffad0729d not found: ID does not exist" containerID="e17180c9f9ce5efa2045df448bf4c1aa9e01544211ef74c31079ed6ffad0729d"
Sep 16 19:04:01.886572 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.886484    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e17180c9f9ce5efa2045df448bf4c1aa9e01544211ef74c31079ed6ffad0729d"} err="failed to get container status \"e17180c9f9ce5efa2045df448bf4c1aa9e01544211ef74c31079ed6ffad0729d\": rpc error: code = NotFound desc = could not find container \"e17180c9f9ce5efa2045df448bf4c1aa9e01544211ef74c31079ed6ffad0729d\": container with ID starting with e17180c9f9ce5efa2045df448bf4c1aa9e01544211ef74c31079ed6ffad0729d not found: ID does not exist"
Sep 16 19:04:01.886572 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.886509    2578 scope.go:117] "RemoveContainer" containerID="5a4475fb2b6cd7bf5291101b215c5c6340e33faf39b4544fabba250563b0c7d0"
Sep 16 19:04:01.886812 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:04:01.886793    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a4475fb2b6cd7bf5291101b215c5c6340e33faf39b4544fabba250563b0c7d0\": container with ID starting with 5a4475fb2b6cd7bf5291101b215c5c6340e33faf39b4544fabba250563b0c7d0 not found: ID does not exist" containerID="5a4475fb2b6cd7bf5291101b215c5c6340e33faf39b4544fabba250563b0c7d0"
Sep 16 19:04:01.886871 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.886822    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a4475fb2b6cd7bf5291101b215c5c6340e33faf39b4544fabba250563b0c7d0"} err="failed to get container status \"5a4475fb2b6cd7bf5291101b215c5c6340e33faf39b4544fabba250563b0c7d0\": rpc error: code = NotFound desc = could not find container \"5a4475fb2b6cd7bf5291101b215c5c6340e33faf39b4544fabba250563b0c7d0\": container with ID starting with 5a4475fb2b6cd7bf5291101b215c5c6340e33faf39b4544fabba250563b0c7d0 not found: ID does not exist"
Sep 16 19:04:01.889472 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.889450    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"]
Sep 16 19:04:01.897511 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:01.897488    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-050f3bde-openshift-default-55dc5d49c6-ws6sv"]
Sep 16 19:04:02.710885 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:02.710853    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5a645e4-76dc-4b52-8ac0-02a25b06af42" path="/var/lib/kubelet/pods/e5a645e4-76dc-4b52-8ac0-02a25b06af42/volumes"
Sep 16 19:04:11.350811 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.350768    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"]
Sep 16 19:04:11.351320 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.351229    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2" containerName="revoke-keys"
Sep 16 19:04:11.351320 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.351242    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2" containerName="revoke-keys"
Sep 16 19:04:11.351320 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.351262    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5a645e4-76dc-4b52-8ac0-02a25b06af42" containerName="istio-proxy"
Sep 16 19:04:11.351320 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.351267    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5a645e4-76dc-4b52-8ac0-02a25b06af42" containerName="istio-proxy"
Sep 16 19:04:11.351320 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.351278    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5a645e4-76dc-4b52-8ac0-02a25b06af42" containerName="istio-proxy"
Sep 16 19:04:11.351320 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.351284    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5a645e4-76dc-4b52-8ac0-02a25b06af42" containerName="istio-proxy"
Sep 16 19:04:11.351644 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.351343    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="56b76fb3-fb63-45b6-ad4e-fbdd47ae1fe2" containerName="revoke-keys"
Sep 16 19:04:11.351644 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.351354    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5a645e4-76dc-4b52-8ac0-02a25b06af42" containerName="istio-proxy"
Sep 16 19:04:11.351644 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.351360    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5a645e4-76dc-4b52-8ac0-02a25b06af42" containerName="istio-proxy"
Sep 16 19:04:11.353559 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.353531    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.356084 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.356061    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-c20219-openshift-default-dockercfg-m928h\""
Sep 16 19:04:11.369644 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.369613    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"]
Sep 16 19:04:11.440421 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.440387    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b163981d-51c3-4f96-9597-228867b865e5-istio-token\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.440421 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.440425    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-workload-certs\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.440656 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.440534    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b163981d-51c3-4f96-9597-228867b865e5-istiod-ca-cert\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.440656 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.440551    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-workload-socket\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.440656 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.440577    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-credential-socket\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.440656 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.440594    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-istio-envoy\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.440656 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.440625    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzjp4\" (UniqueName: \"kubernetes.io/projected/b163981d-51c3-4f96-9597-228867b865e5-kube-api-access-kzjp4\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.440818 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.440710    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b163981d-51c3-4f96-9597-228867b865e5-istio-podinfo\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.440818 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.440776    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-istio-data\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.542198 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.542138    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b163981d-51c3-4f96-9597-228867b865e5-istiod-ca-cert\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.542198 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.542199    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-workload-socket\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.542455 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.542244    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-credential-socket\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.542455 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.542270    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-istio-envoy\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.542455 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.542309    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzjp4\" (UniqueName: \"kubernetes.io/projected/b163981d-51c3-4f96-9597-228867b865e5-kube-api-access-kzjp4\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.542455 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.542364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b163981d-51c3-4f96-9597-228867b865e5-istio-podinfo\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.542455 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.542412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-istio-data\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.542738 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.542471    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b163981d-51c3-4f96-9597-228867b865e5-istio-token\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.542738 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.542500    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-workload-certs\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.542738 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.542636    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-workload-socket\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.542983 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.542962    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b163981d-51c3-4f96-9597-228867b865e5-istiod-ca-cert\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.543347 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.543325    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-workload-certs\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.543465 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.543427    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-istio-data\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.543627 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.543609    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-credential-socket\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.545490 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.545464    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-istio-envoy\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.545612 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.545515    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b163981d-51c3-4f96-9597-228867b865e5-istio-podinfo\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.551152 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.551126    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzjp4\" (UniqueName: \"kubernetes.io/projected/b163981d-51c3-4f96-9597-228867b865e5-kube-api-access-kzjp4\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.551776 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.551756    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b163981d-51c3-4f96-9597-228867b865e5-istio-token\") pod \"e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") " pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.667815 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.667779    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:11.970493 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.970467    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"]
Sep 16 19:04:11.972207 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:04:11.972146    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb163981d_51c3_4f96_9597_228867b865e5.slice/crio-aa7ccd03d9557efc88cd9e3edfdcf0cc86cc197d1f7b1c8d1bd2bdc338e61249 WatchSource:0}: Error finding container aa7ccd03d9557efc88cd9e3edfdcf0cc86cc197d1f7b1c8d1bd2bdc338e61249: Status 404 returned error can't find the container with id aa7ccd03d9557efc88cd9e3edfdcf0cc86cc197d1f7b1c8d1bd2bdc338e61249
Sep 16 19:04:11.974526 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.974491    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:04:11.974601 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.974561    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:04:11.974601 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:11.974589    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:04:12.960683 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:12.960633    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7" event={"ID":"b163981d-51c3-4f96-9597-228867b865e5","Type":"ContainerStarted","Data":"6ff25e74ee38ed95b3932ca204c3e1264cdbdbb40f85c8572801c43bbf5f54a6"}
Sep 16 19:04:12.961167 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:12.960734    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7" event={"ID":"b163981d-51c3-4f96-9597-228867b865e5","Type":"ContainerStarted","Data":"aa7ccd03d9557efc88cd9e3edfdcf0cc86cc197d1f7b1c8d1bd2bdc338e61249"}
Sep 16 19:04:12.986536 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:12.986478    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7" podStartSLOduration=1.986462996 podStartE2EDuration="1.986462996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:04:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:04:12.983467904 +0000 UTC m=+1480.771032086" watchObservedRunningTime="2026-09-16 19:04:12.986462996 +0000 UTC m=+1480.774027151"
Sep 16 19:04:13.652623 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:13.652581    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h"]
Sep 16 19:04:13.655891 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:13.655874    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h"
Sep 16 19:04:13.658437 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:13.658411    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nmr62\""
Sep 16 19:04:13.665527 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:13.665498    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h"]
Sep 16 19:04:13.668137 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:13.668108    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:13.673969 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:13.673936    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:13.766786 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:13.766753    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a81a8154-3d5a-4bcd-90da-b333da393975-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h\" (UID: \"a81a8154-3d5a-4bcd-90da-b333da393975\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h"
Sep 16 19:04:13.867502 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:13.867462    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a81a8154-3d5a-4bcd-90da-b333da393975-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h\" (UID: \"a81a8154-3d5a-4bcd-90da-b333da393975\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h"
Sep 16 19:04:13.868186 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:13.868151    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a81a8154-3d5a-4bcd-90da-b333da393975-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h\" (UID: \"a81a8154-3d5a-4bcd-90da-b333da393975\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h"
Sep 16 19:04:13.967810 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:13.967705    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:13.968746 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:13.968724    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h"
Sep 16 19:04:13.968805 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:13.968782    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:14.256763 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:14.256721    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h"]
Sep 16 19:04:14.257669 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:04:14.257643    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda81a8154_3d5a_4bcd_90da_b333da393975.slice/crio-52a967fb4425584baf3389d1d9d3e09f2ac48728172e034b54da428d306cea32 WatchSource:0}: Error finding container 52a967fb4425584baf3389d1d9d3e09f2ac48728172e034b54da428d306cea32: Status 404 returned error can't find the container with id 52a967fb4425584baf3389d1d9d3e09f2ac48728172e034b54da428d306cea32
Sep 16 19:04:14.977498 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:14.977452    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h" event={"ID":"a81a8154-3d5a-4bcd-90da-b333da393975","Type":"ContainerStarted","Data":"2c9acf783e49ccd60c38162d149bfd4bb531e31b9687b72fde17ac94720b0d6c"}
Sep 16 19:04:14.977912 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:14.977510    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h" event={"ID":"a81a8154-3d5a-4bcd-90da-b333da393975","Type":"ContainerStarted","Data":"52a967fb4425584baf3389d1d9d3e09f2ac48728172e034b54da428d306cea32"}
Sep 16 19:04:15.003064 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:15.003011    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h" podStartSLOduration=2.002995331 podStartE2EDuration="2.002995331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:04:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:04:15.001298107 +0000 UTC m=+1482.788862280" watchObservedRunningTime="2026-09-16 19:04:15.002995331 +0000 UTC m=+1482.790559527"
Sep 16 19:04:15.986505 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:15.986467    2578 generic.go:356] "Generic (PLEG): container finished" podID="a81a8154-3d5a-4bcd-90da-b333da393975" containerID="2c9acf783e49ccd60c38162d149bfd4bb531e31b9687b72fde17ac94720b0d6c" exitCode=0
Sep 16 19:04:15.986939 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:15.986558    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h" event={"ID":"a81a8154-3d5a-4bcd-90da-b333da393975","Type":"ContainerDied","Data":"2c9acf783e49ccd60c38162d149bfd4bb531e31b9687b72fde17ac94720b0d6c"}
Sep 16 19:04:17.137661 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:17.137631    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h"
Sep 16 19:04:17.205134 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:17.205095    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a81a8154-3d5a-4bcd-90da-b333da393975-maas-api-service-ca\") pod \"a81a8154-3d5a-4bcd-90da-b333da393975\" (UID: \"a81a8154-3d5a-4bcd-90da-b333da393975\") "
Sep 16 19:04:17.205465 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:17.205441    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a81a8154-3d5a-4bcd-90da-b333da393975-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a81a8154-3d5a-4bcd-90da-b333da393975" (UID: "a81a8154-3d5a-4bcd-90da-b333da393975"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:04:17.306164 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:17.306074    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a81a8154-3d5a-4bcd-90da-b333da393975-maas-api-service-ca\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:04:18.002598 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:18.002568    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h"
Sep 16 19:04:18.002776 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:18.002602    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h" event={"ID":"a81a8154-3d5a-4bcd-90da-b333da393975","Type":"ContainerDied","Data":"52a967fb4425584baf3389d1d9d3e09f2ac48728172e034b54da428d306cea32"}
Sep 16 19:04:18.002776 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:18.002649    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52a967fb4425584baf3389d1d9d3e09f2ac48728172e034b54da428d306cea32"
Sep 16 19:04:24.840631 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:24.840528    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h"]
Sep 16 19:04:24.848378 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:24.848341    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-9d2450-tdz2h"]
Sep 16 19:04:26.710393 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:26.710352    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a81a8154-3d5a-4bcd-90da-b333da393975" path="/var/lib/kubelet/pods/a81a8154-3d5a-4bcd-90da-b333da393975/volumes"
Sep 16 19:04:32.782131 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:32.782104    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 19:04:32.783614 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:32.783586    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 19:04:32.788534 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:32.788511    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 19:04:32.789811 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:32.789790    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 19:04:35.471641 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.471605    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"]
Sep 16 19:04:35.472064 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.472051    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a81a8154-3d5a-4bcd-90da-b333da393975" containerName="revoke-keys"
Sep 16 19:04:35.472117 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.472065    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a81a8154-3d5a-4bcd-90da-b333da393975" containerName="revoke-keys"
Sep 16 19:04:35.472158 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.472139    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a81a8154-3d5a-4bcd-90da-b333da393975" containerName="revoke-keys"
Sep 16 19:04:35.475837 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.475817    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.481962 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.481927    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-58bf51-openshift-default-dockercfg-bhpqp\""
Sep 16 19:04:35.512427 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.512390    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"]
Sep 16 19:04:35.605194 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.604224    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-credential-socket\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.605194 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.604289    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b46a4a50-fafe-4187-b140-220649dd54c0-istiod-ca-cert\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.605194 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.604326    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-workload-socket\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.605194 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.604403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l76h2\" (UniqueName: \"kubernetes.io/projected/b46a4a50-fafe-4187-b140-220649dd54c0-kube-api-access-l76h2\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.605194 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.604468    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-istio-envoy\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.605194 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.604523    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-workload-certs\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.605194 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.604555    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b46a4a50-fafe-4187-b140-220649dd54c0-istio-token\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.605194 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.604661    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-istio-data\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.605194 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.604691    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b46a4a50-fafe-4187-b140-220649dd54c0-istio-podinfo\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.706043 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.706007    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b46a4a50-fafe-4187-b140-220649dd54c0-istio-podinfo\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.706263 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.706055    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-credential-socket\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.706263 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.706092    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b46a4a50-fafe-4187-b140-220649dd54c0-istiod-ca-cert\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.706263 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.706123    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-workload-socket\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.706263 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.706206    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l76h2\" (UniqueName: \"kubernetes.io/projected/b46a4a50-fafe-4187-b140-220649dd54c0-kube-api-access-l76h2\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.706491 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.706269    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-istio-envoy\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.706491 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.706322    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-workload-certs\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.706491 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.706354    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b46a4a50-fafe-4187-b140-220649dd54c0-istio-token\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.706491 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.706449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-istio-data\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.706710 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.706620    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-workload-socket\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.706840 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.706811    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-istio-data\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.707120 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.706903    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-workload-certs\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.707120 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.706990    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b46a4a50-fafe-4187-b140-220649dd54c0-istiod-ca-cert\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.707284 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.707110    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-credential-socket\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.709528 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.709499    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-istio-envoy\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.709830 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.709808    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b46a4a50-fafe-4187-b140-220649dd54c0-istio-podinfo\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.718297 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.718273    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b46a4a50-fafe-4187-b140-220649dd54c0-istio-token\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.718538 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.718511    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l76h2\" (UniqueName: \"kubernetes.io/projected/b46a4a50-fafe-4187-b140-220649dd54c0-kube-api-access-l76h2\") pod \"e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") " pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:35.786609 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:35.786520    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:36.079071 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:36.078980    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"]
Sep 16 19:04:36.097510 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:36.096007    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:04:36.097510 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:36.096103    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:04:36.097510 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:36.096145    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:04:36.128260 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:36.128223    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2" event={"ID":"b46a4a50-fafe-4187-b140-220649dd54c0","Type":"ContainerStarted","Data":"bdccff54a5c6f7f44c0bf468c75eeb52e9ad41ab917a224343fa1a69b731ef36"}
Sep 16 19:04:37.136865 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:37.136823    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2" event={"ID":"b46a4a50-fafe-4187-b140-220649dd54c0","Type":"ContainerStarted","Data":"a7fa671fbdee4c2190817b6175e3acec8828d61b33886492d38b406d595b3fbb"}
Sep 16 19:04:37.161524 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:37.161472    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2" podStartSLOduration=2.161455622 podStartE2EDuration="2.161455622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:04:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:04:37.159463096 +0000 UTC m=+1504.947027282" watchObservedRunningTime="2026-09-16 19:04:37.161455622 +0000 UTC m=+1504.949019777"
Sep 16 19:04:37.787001 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:37.786959    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:37.792771 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:37.792746    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:38.145008 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:38.144979    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:38.146044 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:38.146023    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:04:38.671691 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:38.671656    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"]
Sep 16 19:04:53.691288 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:53.691238    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc" podUID="5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5" containerName="main" containerID="cri-o://fc00e541215231c75732f553931f753c2cbc9779712ce03ec9f89124c9a46217" gracePeriod=45
Sep 16 19:04:53.951448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:53.951379    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:04:54.012781 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.012741    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-tls-certs\") pod \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") "
Sep 16 19:04:54.012980 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.012846    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-model-cache\") pod \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") "
Sep 16 19:04:54.012980 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.012886    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-home\") pod \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") "
Sep 16 19:04:54.012980 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.012937    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67mgm\" (UniqueName: \"kubernetes.io/projected/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-kube-api-access-67mgm\") pod \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") "
Sep 16 19:04:54.013149 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.012989    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-dshm\") pod \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") "
Sep 16 19:04:54.013149 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.013020    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-tmp-dir\") pod \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\" (UID: \"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5\") "
Sep 16 19:04:54.013313 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.013233    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-model-cache" (OuterVolumeSpecName: "model-cache") pod "5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5" (UID: "5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:04:54.013313 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.013250    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-home" (OuterVolumeSpecName: "home") pod "5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5" (UID: "5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:04:54.013468 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.013451    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5" (UID: "5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:04:54.013644 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.013621    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-model-cache\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:04:54.013841 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.013803    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-home\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:04:54.013841 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.013821    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-tmp-dir\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:04:54.015593 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.015545    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-dshm" (OuterVolumeSpecName: "dshm") pod "5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5" (UID: "5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:04:54.015702 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.015670    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-kube-api-access-67mgm" (OuterVolumeSpecName: "kube-api-access-67mgm") pod "5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5" (UID: "5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5"). InnerVolumeSpecName "kube-api-access-67mgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:04:54.015752 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.015729    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5" (UID: "5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:04:54.114656 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.114617    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-dshm\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:04:54.114656 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.114650    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-tls-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:04:54.114656 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.114662    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67mgm\" (UniqueName: \"kubernetes.io/projected/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5-kube-api-access-67mgm\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:04:54.268033 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.267936    2578 generic.go:356] "Generic (PLEG): container finished" podID="5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5" containerID="fc00e541215231c75732f553931f753c2cbc9779712ce03ec9f89124c9a46217" exitCode=0
Sep 16 19:04:54.268033 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.268001    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc" event={"ID":"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5","Type":"ContainerDied","Data":"fc00e541215231c75732f553931f753c2cbc9779712ce03ec9f89124c9a46217"}
Sep 16 19:04:54.268308 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.268052    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc" event={"ID":"5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5","Type":"ContainerDied","Data":"f4ce502e1ae79cc633b0e395c41f55cb0f807415e1ae80b2bf1a351a9e667e96"}
Sep 16 19:04:54.268308 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.268054    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"
Sep 16 19:04:54.268308 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.268074    2578 scope.go:117] "RemoveContainer" containerID="fc00e541215231c75732f553931f753c2cbc9779712ce03ec9f89124c9a46217"
Sep 16 19:04:54.279550 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.279527    2578 scope.go:117] "RemoveContainer" containerID="fc00e541215231c75732f553931f753c2cbc9779712ce03ec9f89124c9a46217"
Sep 16 19:04:54.279819 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:04:54.279799    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc00e541215231c75732f553931f753c2cbc9779712ce03ec9f89124c9a46217\": container with ID starting with fc00e541215231c75732f553931f753c2cbc9779712ce03ec9f89124c9a46217 not found: ID does not exist" containerID="fc00e541215231c75732f553931f753c2cbc9779712ce03ec9f89124c9a46217"
Sep 16 19:04:54.279888 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.279827    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc00e541215231c75732f553931f753c2cbc9779712ce03ec9f89124c9a46217"} err="failed to get container status \"fc00e541215231c75732f553931f753c2cbc9779712ce03ec9f89124c9a46217\": rpc error: code = NotFound desc = could not find container \"fc00e541215231c75732f553931f753c2cbc9779712ce03ec9f89124c9a46217\": container with ID starting with fc00e541215231c75732f553931f753c2cbc9779712ce03ec9f89124c9a46217 not found: ID does not exist"
Sep 16 19:04:54.302384 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.302350    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"]
Sep 16 19:04:54.305300 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.305270    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-ec32e7/test-model-ec32e7-kserve-7ff6d69c9-zdlqc"]
Sep 16 19:04:54.711784 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:54.711750    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5" path="/var/lib/kubelet/pods/5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5/volumes"
Sep 16 19:04:57.291661 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:57.291627    2578 generic.go:356] "Generic (PLEG): container finished" podID="b163981d-51c3-4f96-9597-228867b865e5" containerID="6ff25e74ee38ed95b3932ca204c3e1264cdbdbb40f85c8572801c43bbf5f54a6" exitCode=0
Sep 16 19:04:57.292091 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:57.291702    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7" event={"ID":"b163981d-51c3-4f96-9597-228867b865e5","Type":"ContainerDied","Data":"6ff25e74ee38ed95b3932ca204c3e1264cdbdbb40f85c8572801c43bbf5f54a6"}
Sep 16 19:04:57.292258 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:57.292226    2578 scope.go:117] "RemoveContainer" containerID="6ff25e74ee38ed95b3932ca204c3e1264cdbdbb40f85c8572801c43bbf5f54a6"
Sep 16 19:04:57.293482 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:57.293449    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:04:57.293547 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:57.293520    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:04:57.293585 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:57.293556    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:04:58.300692 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:58.300653    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7" event={"ID":"b163981d-51c3-4f96-9597-228867b865e5","Type":"ContainerStarted","Data":"ea2c8bb6092a65469cd9f4f4f69fe129da3eb452bda67a072956de35785b8d2b"}
Sep 16 19:04:58.668403 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:58.668361    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:58.669674 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:58.669643    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 16 19:04:58.669819 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:58.669699    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7" podUID="b163981d-51c3-4f96-9597-228867b865e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 16 19:04:58.968196 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:58.968095    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:04:59.668329 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:59.668287    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 16 19:04:59.668722 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:04:59.668351    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7" podUID="b163981d-51c3-4f96-9597-228867b865e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 16 19:05:00.668343 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:00.668306    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 16 19:05:00.668762 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:00.668367    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7" podUID="b163981d-51c3-4f96-9597-228867b865e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 16 19:05:01.668610 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:01.668571    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 16 19:05:01.669019 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:01.668638    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7" podUID="b163981d-51c3-4f96-9597-228867b865e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 16 19:05:02.668315 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:02.668278    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 16 19:05:02.668505 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:02.668339    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7" podUID="b163981d-51c3-4f96-9597-228867b865e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 16 19:05:03.691751 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:03.691712    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:05:03.692985 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:03.692964    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:05:08.618078 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:08.618042    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"]
Sep 16 19:05:08.618642 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:08.618271    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d" podUID="2582bb11-32ae-486b-bae6-65b64397fba6" containerName="istio-proxy" containerID="cri-o://3ee8d6366f5a9332050ec5e0f0a6c3ebfbd74b8665d81ec34203465d7e82cf52" gracePeriod=30
Sep 16 19:05:09.368917 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.368876    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"]
Sep 16 19:05:09.369905 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.369875    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5" containerName="main"
Sep 16 19:05:09.370080 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.370068    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5" containerName="main"
Sep 16 19:05:09.370360 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.370344    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a51d474-0d49-4e6b-b5d0-e4c5bb59a0f5" containerName="main"
Sep 16 19:05:09.379205 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.377105    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.380878 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.380855    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-c190a0-openshift-default-dockercfg-hfkfl\""
Sep 16 19:05:09.399420 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.399336    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"]
Sep 16 19:05:09.492771 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.492729    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-workload-certs\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.493072 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.493053    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-credential-socket\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.493298 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.493280    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c1295f7-3e69-4025-9381-fa348f881123-istio-token\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.493468 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.493453    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-istio-envoy\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.493604 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.493591    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c1295f7-3e69-4025-9381-fa348f881123-istiod-ca-cert\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.493756 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.493742    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-istio-data\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.493919 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.493906    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c1295f7-3e69-4025-9381-fa348f881123-istio-podinfo\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.494060 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.494044    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmg7l\" (UniqueName: \"kubernetes.io/projected/5c1295f7-3e69-4025-9381-fa348f881123-kube-api-access-mmg7l\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.494235 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.494220    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-workload-socket\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.594977 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.594945    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-workload-certs\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.595551 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.595534    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-credential-socket\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.595805 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.595780    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c1295f7-3e69-4025-9381-fa348f881123-istio-token\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.595927 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.595908    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-credential-socket\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.596013 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.595489    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-workload-certs\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.596076 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.596009    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-istio-envoy\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.596076 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.596048    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c1295f7-3e69-4025-9381-fa348f881123-istiod-ca-cert\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.596202 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.596100    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-istio-data\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.596405 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.596387    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-istio-data\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.596529 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.596499    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c1295f7-3e69-4025-9381-fa348f881123-istio-podinfo\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.596601 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.596567    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmg7l\" (UniqueName: \"kubernetes.io/projected/5c1295f7-3e69-4025-9381-fa348f881123-kube-api-access-mmg7l\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.596701 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.596683    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-workload-socket\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.597078 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.597018    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c1295f7-3e69-4025-9381-fa348f881123-istiod-ca-cert\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.597078 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.597038    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-workload-socket\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.598495 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.598452    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-istio-envoy\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.599079 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.599062    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c1295f7-3e69-4025-9381-fa348f881123-istio-podinfo\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.610084 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.610056    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmg7l\" (UniqueName: \"kubernetes.io/projected/5c1295f7-3e69-4025-9381-fa348f881123-kube-api-access-mmg7l\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.610625 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.610606    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c1295f7-3e69-4025-9381-fa348f881123-istio-token\") pod \"e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") " pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.619651 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.619576    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn"]
Sep 16 19:05:09.622852 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.622831    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn"
Sep 16 19:05:09.625275 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.625258    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nmr62\""
Sep 16 19:05:09.630739 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.630667    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn"]
Sep 16 19:05:09.696734 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.696451    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:09.699233 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.699143    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e05e527f-b9b2-487e-bfee-a931d7b4843a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn\" (UID: \"e05e527f-b9b2-487e-bfee-a931d7b4843a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn"
Sep 16 19:05:09.802272 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.802235    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e05e527f-b9b2-487e-bfee-a931d7b4843a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn\" (UID: \"e05e527f-b9b2-487e-bfee-a931d7b4843a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn"
Sep 16 19:05:09.803578 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.803187    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e05e527f-b9b2-487e-bfee-a931d7b4843a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn\" (UID: \"e05e527f-b9b2-487e-bfee-a931d7b4843a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn"
Sep 16 19:05:09.969247 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:09.964891    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn"
Sep 16 19:05:10.021797 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:10.021762    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"]
Sep 16 19:05:10.023466 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:05:10.023230    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c1295f7_3e69_4025_9381_fa348f881123.slice/crio-278364c89a6acbfe6157d185115eaefb4fa07d7331b226a3a71287e0ced303bb WatchSource:0}: Error finding container 278364c89a6acbfe6157d185115eaefb4fa07d7331b226a3a71287e0ced303bb: Status 404 returned error can't find the container with id 278364c89a6acbfe6157d185115eaefb4fa07d7331b226a3a71287e0ced303bb
Sep 16 19:05:10.025849 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:10.025709    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:05:10.025849 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:10.025799    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:05:10.025849 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:10.025847    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:05:10.255195 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:10.255084    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn"]
Sep 16 19:05:10.259776 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:05:10.259727    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode05e527f_b9b2_487e_bfee_a931d7b4843a.slice/crio-58ecdf334fd46564a2a73d7ae16e4234d587a7212708fd4b98169152967a779a WatchSource:0}: Error finding container 58ecdf334fd46564a2a73d7ae16e4234d587a7212708fd4b98169152967a779a: Status 404 returned error can't find the container with id 58ecdf334fd46564a2a73d7ae16e4234d587a7212708fd4b98169152967a779a
Sep 16 19:05:10.393025 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:10.392971    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn" event={"ID":"e05e527f-b9b2-487e-bfee-a931d7b4843a","Type":"ContainerStarted","Data":"58ecdf334fd46564a2a73d7ae16e4234d587a7212708fd4b98169152967a779a"}
Sep 16 19:05:10.395088 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:10.395053    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg" event={"ID":"5c1295f7-3e69-4025-9381-fa348f881123","Type":"ContainerStarted","Data":"f70ccdbcb4556c24c3d22797c2a26cab5abb46a7ebfd682623f629f099c7c26f"}
Sep 16 19:05:10.395281 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:10.395106    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg" event={"ID":"5c1295f7-3e69-4025-9381-fa348f881123","Type":"ContainerStarted","Data":"278364c89a6acbfe6157d185115eaefb4fa07d7331b226a3a71287e0ced303bb"}
Sep 16 19:05:10.422865 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:10.422809    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg" podStartSLOduration=1.422789697 podStartE2EDuration="1.422789697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:05:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:05:10.421714067 +0000 UTC m=+1538.209278273" watchObservedRunningTime="2026-09-16 19:05:10.422789697 +0000 UTC m=+1538.210353852"
Sep 16 19:05:10.698822 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:10.698788    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:11.403637 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:11.403601    2578 generic.go:356] "Generic (PLEG): container finished" podID="e05e527f-b9b2-487e-bfee-a931d7b4843a" containerID="3d60d4475f450a10262d8d2acfeab39227efe1eb5b0b0dc757a140ccf8eaa84e" exitCode=0
Sep 16 19:05:11.403840 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:11.403640    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn" event={"ID":"e05e527f-b9b2-487e-bfee-a931d7b4843a","Type":"ContainerDied","Data":"3d60d4475f450a10262d8d2acfeab39227efe1eb5b0b0dc757a140ccf8eaa84e"}
Sep 16 19:05:11.703982 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:11.703951    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:12.410098 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:12.410061    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:12.411159 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:12.411140    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:05:12.634998 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:12.634971    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn"
Sep 16 19:05:12.736269 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:12.736149    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e05e527f-b9b2-487e-bfee-a931d7b4843a-maas-api-service-ca\") pod \"e05e527f-b9b2-487e-bfee-a931d7b4843a\" (UID: \"e05e527f-b9b2-487e-bfee-a931d7b4843a\") "
Sep 16 19:05:12.736625 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:12.736487    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e05e527f-b9b2-487e-bfee-a931d7b4843a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e05e527f-b9b2-487e-bfee-a931d7b4843a" (UID: "e05e527f-b9b2-487e-bfee-a931d7b4843a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:05:12.838034 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:12.837978    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e05e527f-b9b2-487e-bfee-a931d7b4843a-maas-api-service-ca\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:13.418999 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:13.418965    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn" event={"ID":"e05e527f-b9b2-487e-bfee-a931d7b4843a","Type":"ContainerDied","Data":"58ecdf334fd46564a2a73d7ae16e4234d587a7212708fd4b98169152967a779a"}
Sep 16 19:05:13.419245 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:13.418996    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn"
Sep 16 19:05:13.419358 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:13.419344    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58ecdf334fd46564a2a73d7ae16e4234d587a7212708fd4b98169152967a779a"
Sep 16 19:05:13.932371 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:13.932339    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:05:14.073721 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.073595    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2582bb11-32ae-486b-bae6-65b64397fba6-istio-podinfo\") pod \"2582bb11-32ae-486b-bae6-65b64397fba6\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") "
Sep 16 19:05:14.073721 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.073639    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-credential-socket\") pod \"2582bb11-32ae-486b-bae6-65b64397fba6\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") "
Sep 16 19:05:14.073721 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.073694    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-workload-socket\") pod \"2582bb11-32ae-486b-bae6-65b64397fba6\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") "
Sep 16 19:05:14.074011 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.073741    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvlk4\" (UniqueName: \"kubernetes.io/projected/2582bb11-32ae-486b-bae6-65b64397fba6-kube-api-access-kvlk4\") pod \"2582bb11-32ae-486b-bae6-65b64397fba6\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") "
Sep 16 19:05:14.074011 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.073788    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-istio-envoy\") pod \"2582bb11-32ae-486b-bae6-65b64397fba6\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") "
Sep 16 19:05:14.074011 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.073843    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2582bb11-32ae-486b-bae6-65b64397fba6-istio-token\") pod \"2582bb11-32ae-486b-bae6-65b64397fba6\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") "
Sep 16 19:05:14.074011 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.073863    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-workload-certs\") pod \"2582bb11-32ae-486b-bae6-65b64397fba6\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") "
Sep 16 19:05:14.074011 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.073933    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2582bb11-32ae-486b-bae6-65b64397fba6-istiod-ca-cert\") pod \"2582bb11-32ae-486b-bae6-65b64397fba6\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") "
Sep 16 19:05:14.074011 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.073933    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2582bb11-32ae-486b-bae6-65b64397fba6" (UID: "2582bb11-32ae-486b-bae6-65b64397fba6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:14.074011 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.073961    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-istio-data\") pod \"2582bb11-32ae-486b-bae6-65b64397fba6\" (UID: \"2582bb11-32ae-486b-bae6-65b64397fba6\") "
Sep 16 19:05:14.074934 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.074615    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2582bb11-32ae-486b-bae6-65b64397fba6" (UID: "2582bb11-32ae-486b-bae6-65b64397fba6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:14.074934 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.074686    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-istio-data" (OuterVolumeSpecName: "istio-data") pod "2582bb11-32ae-486b-bae6-65b64397fba6" (UID: "2582bb11-32ae-486b-bae6-65b64397fba6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:14.074934 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.074715    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2582bb11-32ae-486b-bae6-65b64397fba6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2582bb11-32ae-486b-bae6-65b64397fba6" (UID: "2582bb11-32ae-486b-bae6-65b64397fba6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:05:14.074934 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.074857    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2582bb11-32ae-486b-bae6-65b64397fba6" (UID: "2582bb11-32ae-486b-bae6-65b64397fba6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:14.077669 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.076365    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2582bb11-32ae-486b-bae6-65b64397fba6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2582bb11-32ae-486b-bae6-65b64397fba6" (UID: "2582bb11-32ae-486b-bae6-65b64397fba6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:05:14.077669 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.076443    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2582bb11-32ae-486b-bae6-65b64397fba6-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:14.077669 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.076477    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:14.077669 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.076492    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:14.077669 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.076505    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:14.077669 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.076519    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:14.086491 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.086464    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2582bb11-32ae-486b-bae6-65b64397fba6-kube-api-access-kvlk4" (OuterVolumeSpecName: "kube-api-access-kvlk4") pod "2582bb11-32ae-486b-bae6-65b64397fba6" (UID: "2582bb11-32ae-486b-bae6-65b64397fba6"). InnerVolumeSpecName "kube-api-access-kvlk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:14.086491 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.086427    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2582bb11-32ae-486b-bae6-65b64397fba6-istio-token" (OuterVolumeSpecName: "istio-token") pod "2582bb11-32ae-486b-bae6-65b64397fba6" (UID: "2582bb11-32ae-486b-bae6-65b64397fba6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:14.087779 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.087755    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2582bb11-32ae-486b-bae6-65b64397fba6" (UID: "2582bb11-32ae-486b-bae6-65b64397fba6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:14.130700 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.130666    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7223-c1b582"]
Sep 16 19:05:14.131134 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.131121    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2582bb11-32ae-486b-bae6-65b64397fba6" containerName="istio-proxy"
Sep 16 19:05:14.131217 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.131135    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2582bb11-32ae-486b-bae6-65b64397fba6" containerName="istio-proxy"
Sep 16 19:05:14.131217 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.131146    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2582bb11-32ae-486b-bae6-65b64397fba6" containerName="istio-proxy"
Sep 16 19:05:14.131217 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.131152    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2582bb11-32ae-486b-bae6-65b64397fba6" containerName="istio-proxy"
Sep 16 19:05:14.131217 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.131167    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e05e527f-b9b2-487e-bfee-a931d7b4843a" containerName="revoke-keys"
Sep 16 19:05:14.131217 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.131185    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e05e527f-b9b2-487e-bfee-a931d7b4843a" containerName="revoke-keys"
Sep 16 19:05:14.131380 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.131256    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2582bb11-32ae-486b-bae6-65b64397fba6" containerName="istio-proxy"
Sep 16 19:05:14.131380 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.131264    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e05e527f-b9b2-487e-bfee-a931d7b4843a" containerName="revoke-keys"
Sep 16 19:05:14.133514 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.133493    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7223-c1b582"
Sep 16 19:05:14.140068 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.140044    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-q2jrz\""
Sep 16 19:05:14.156574 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.156523    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7223-c1b582"]
Sep 16 19:05:14.177708 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.177681    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvlk4\" (UniqueName: \"kubernetes.io/projected/2582bb11-32ae-486b-bae6-65b64397fba6-kube-api-access-kvlk4\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:14.177708 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.177707    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2582bb11-32ae-486b-bae6-65b64397fba6-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:14.177864 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.177717    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2582bb11-32ae-486b-bae6-65b64397fba6-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:14.177864 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.177726    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2582bb11-32ae-486b-bae6-65b64397fba6-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:14.278714 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.278678    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7zxm\" (UniqueName: \"kubernetes.io/projected/4a2c05c5-f132-44b8-826b-6547551ad7c8-kube-api-access-v7zxm\") pod \"test-curl-7223-c1b582\" (UID: \"4a2c05c5-f132-44b8-826b-6547551ad7c8\") " pod="opendatahub/test-curl-7223-c1b582"
Sep 16 19:05:14.380197 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.380152    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7zxm\" (UniqueName: \"kubernetes.io/projected/4a2c05c5-f132-44b8-826b-6547551ad7c8-kube-api-access-v7zxm\") pod \"test-curl-7223-c1b582\" (UID: \"4a2c05c5-f132-44b8-826b-6547551ad7c8\") " pod="opendatahub/test-curl-7223-c1b582"
Sep 16 19:05:14.394903 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.394874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7zxm\" (UniqueName: \"kubernetes.io/projected/4a2c05c5-f132-44b8-826b-6547551ad7c8-kube-api-access-v7zxm\") pod \"test-curl-7223-c1b582\" (UID: \"4a2c05c5-f132-44b8-826b-6547551ad7c8\") " pod="opendatahub/test-curl-7223-c1b582"
Sep 16 19:05:14.432379 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.432345    2578 generic.go:356] "Generic (PLEG): container finished" podID="2582bb11-32ae-486b-bae6-65b64397fba6" containerID="3ee8d6366f5a9332050ec5e0f0a6c3ebfbd74b8665d81ec34203465d7e82cf52" exitCode=0
Sep 16 19:05:14.432523 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.432411    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"
Sep 16 19:05:14.432523 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.432433    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d" event={"ID":"2582bb11-32ae-486b-bae6-65b64397fba6","Type":"ContainerDied","Data":"3ee8d6366f5a9332050ec5e0f0a6c3ebfbd74b8665d81ec34203465d7e82cf52"}
Sep 16 19:05:14.432523 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.432490    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d" event={"ID":"2582bb11-32ae-486b-bae6-65b64397fba6","Type":"ContainerDied","Data":"23769da7219e41641030759fa40d746abdf9b8219c145b8bdc8e7e4b48a70ea4"}
Sep 16 19:05:14.432523 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.432517    2578 scope.go:117] "RemoveContainer" containerID="3ee8d6366f5a9332050ec5e0f0a6c3ebfbd74b8665d81ec34203465d7e82cf52"
Sep 16 19:05:14.444988 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.444967    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7223-c1b582"
Sep 16 19:05:14.445571 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.445551    2578 scope.go:117] "RemoveContainer" containerID="ca86029fd67c378a226b7c47ecc8cbbb63c51f1caa43ef76b8f321e00ef8b9a2"
Sep 16 19:05:14.460900 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.460880    2578 scope.go:117] "RemoveContainer" containerID="3ee8d6366f5a9332050ec5e0f0a6c3ebfbd74b8665d81ec34203465d7e82cf52"
Sep 16 19:05:14.461241 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:05:14.461215    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ee8d6366f5a9332050ec5e0f0a6c3ebfbd74b8665d81ec34203465d7e82cf52\": container with ID starting with 3ee8d6366f5a9332050ec5e0f0a6c3ebfbd74b8665d81ec34203465d7e82cf52 not found: ID does not exist" containerID="3ee8d6366f5a9332050ec5e0f0a6c3ebfbd74b8665d81ec34203465d7e82cf52"
Sep 16 19:05:14.461339 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.461248    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ee8d6366f5a9332050ec5e0f0a6c3ebfbd74b8665d81ec34203465d7e82cf52"} err="failed to get container status \"3ee8d6366f5a9332050ec5e0f0a6c3ebfbd74b8665d81ec34203465d7e82cf52\": rpc error: code = NotFound desc = could not find container \"3ee8d6366f5a9332050ec5e0f0a6c3ebfbd74b8665d81ec34203465d7e82cf52\": container with ID starting with 3ee8d6366f5a9332050ec5e0f0a6c3ebfbd74b8665d81ec34203465d7e82cf52 not found: ID does not exist"
Sep 16 19:05:14.461339 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.461267    2578 scope.go:117] "RemoveContainer" containerID="ca86029fd67c378a226b7c47ecc8cbbb63c51f1caa43ef76b8f321e00ef8b9a2"
Sep 16 19:05:14.461579 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:05:14.461562    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca86029fd67c378a226b7c47ecc8cbbb63c51f1caa43ef76b8f321e00ef8b9a2\": container with ID starting with ca86029fd67c378a226b7c47ecc8cbbb63c51f1caa43ef76b8f321e00ef8b9a2 not found: ID does not exist" containerID="ca86029fd67c378a226b7c47ecc8cbbb63c51f1caa43ef76b8f321e00ef8b9a2"
Sep 16 19:05:14.461643 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.461589    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca86029fd67c378a226b7c47ecc8cbbb63c51f1caa43ef76b8f321e00ef8b9a2"} err="failed to get container status \"ca86029fd67c378a226b7c47ecc8cbbb63c51f1caa43ef76b8f321e00ef8b9a2\": rpc error: code = NotFound desc = could not find container \"ca86029fd67c378a226b7c47ecc8cbbb63c51f1caa43ef76b8f321e00ef8b9a2\": container with ID starting with ca86029fd67c378a226b7c47ecc8cbbb63c51f1caa43ef76b8f321e00ef8b9a2 not found: ID does not exist"
Sep 16 19:05:14.471979 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.471955    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"]
Sep 16 19:05:14.479239 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.479213    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-9d2450-openshift-default-85c56c7684-5l84d"]
Sep 16 19:05:14.713070 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.712998    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2582bb11-32ae-486b-bae6-65b64397fba6" path="/var/lib/kubelet/pods/2582bb11-32ae-486b-bae6-65b64397fba6/volumes"
Sep 16 19:05:14.727115 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.727088    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7223-c1b582"]
Sep 16 19:05:14.727303 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:05:14.727263    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a2c05c5_f132_44b8_826b_6547551ad7c8.slice/crio-1010256c139eb6d0f62fa52efe33602282620a2207c87ce6abd519664eac9d61 WatchSource:0}: Error finding container 1010256c139eb6d0f62fa52efe33602282620a2207c87ce6abd519664eac9d61: Status 404 returned error can't find the container with id 1010256c139eb6d0f62fa52efe33602282620a2207c87ce6abd519664eac9d61
Sep 16 19:05:14.729305 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:14.729286    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 19:05:15.446053 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:15.445966    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7223-c1b582" event={"ID":"4a2c05c5-f132-44b8-826b-6547551ad7c8","Type":"ContainerStarted","Data":"1010256c139eb6d0f62fa52efe33602282620a2207c87ce6abd519664eac9d61"}
Sep 16 19:05:17.480201 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:17.480133    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7223-c1b582" event={"ID":"4a2c05c5-f132-44b8-826b-6547551ad7c8","Type":"ContainerStarted","Data":"19d836ecb512c97c4f3dabd6954105701ff03b440d895c75641ad2f6b84867ef"}
Sep 16 19:05:17.502922 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:17.502871    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7223-c1b582" podStartSLOduration=1.606625223 podStartE2EDuration="3.502855069s" totalImagesPullingTime="1.896229846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:05:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:05:17.498482712 +0000 UTC m=+1545.286046864" watchObservedRunningTime="2026-09-16 19:05:17.502855069 +0000 UTC m=+1545.290419269"
Sep 16 19:05:18.142288 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.142245    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7223-c1b582"]
Sep 16 19:05:18.148353 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.148318    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7223-c1b582"]
Sep 16 19:05:18.267966 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.267887    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7223-92ea32"]
Sep 16 19:05:18.268393 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.268378    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a2c05c5-f132-44b8-826b-6547551ad7c8" containerName="test-curl-7223-c1b582"
Sep 16 19:05:18.268445 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.268394    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a2c05c5-f132-44b8-826b-6547551ad7c8" containerName="test-curl-7223-c1b582"
Sep 16 19:05:18.268481 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.268463    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a2c05c5-f132-44b8-826b-6547551ad7c8" containerName="test-curl-7223-c1b582"
Sep 16 19:05:18.268481 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.268474    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2582bb11-32ae-486b-bae6-65b64397fba6" containerName="istio-proxy"
Sep 16 19:05:18.270762 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.270747    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7223-92ea32"
Sep 16 19:05:18.280102 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.280073    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7223-92ea32"]
Sep 16 19:05:18.422897 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.422864    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49vt4\" (UniqueName: \"kubernetes.io/projected/8660d141-86ac-4a78-befe-971a0a60e53b-kube-api-access-49vt4\") pod \"test-curl-7223-92ea32\" (UID: \"8660d141-86ac-4a78-befe-971a0a60e53b\") " pod="opendatahub/test-curl-7223-92ea32"
Sep 16 19:05:18.489891 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.489855    2578 generic.go:356] "Generic (PLEG): container finished" podID="b46a4a50-fafe-4187-b140-220649dd54c0" containerID="a7fa671fbdee4c2190817b6175e3acec8828d61b33886492d38b406d595b3fbb" exitCode=0
Sep 16 19:05:18.490377 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.489945    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2" event={"ID":"b46a4a50-fafe-4187-b140-220649dd54c0","Type":"ContainerDied","Data":"a7fa671fbdee4c2190817b6175e3acec8828d61b33886492d38b406d595b3fbb"}
Sep 16 19:05:18.490535 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.490513    2578 scope.go:117] "RemoveContainer" containerID="a7fa671fbdee4c2190817b6175e3acec8828d61b33886492d38b406d595b3fbb"
Sep 16 19:05:18.490773 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:05:18.490707    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7223-c1b582" container="test-curl-7223-c1b582"
Sep 16 19:05:18.493146 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.492044    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:05:18.493146 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.492139    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:05:18.493146 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.492209    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:05:18.524042 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.523980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49vt4\" (UniqueName: \"kubernetes.io/projected/8660d141-86ac-4a78-befe-971a0a60e53b-kube-api-access-49vt4\") pod \"test-curl-7223-92ea32\" (UID: \"8660d141-86ac-4a78-befe-971a0a60e53b\") " pod="opendatahub/test-curl-7223-92ea32"
Sep 16 19:05:18.536270 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.536205    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49vt4\" (UniqueName: \"kubernetes.io/projected/8660d141-86ac-4a78-befe-971a0a60e53b-kube-api-access-49vt4\") pod \"test-curl-7223-92ea32\" (UID: \"8660d141-86ac-4a78-befe-971a0a60e53b\") " pod="opendatahub/test-curl-7223-92ea32"
Sep 16 19:05:18.581651 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.581621    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7223-92ea32"
Sep 16 19:05:18.878574 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:05:18.878543    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8660d141_86ac_4a78_befe_971a0a60e53b.slice/crio-71a55e2d59241e8af00025e6b9beee9a9be4a7834cb13d50028acb3b6d4a3374 WatchSource:0}: Error finding container 71a55e2d59241e8af00025e6b9beee9a9be4a7834cb13d50028acb3b6d4a3374: Status 404 returned error can't find the container with id 71a55e2d59241e8af00025e6b9beee9a9be4a7834cb13d50028acb3b6d4a3374
Sep 16 19:05:18.878787 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:18.878762    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7223-92ea32"]
Sep 16 19:05:19.504286 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:19.504247    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2" event={"ID":"b46a4a50-fafe-4187-b140-220649dd54c0","Type":"ContainerStarted","Data":"79ca899248d49e8e0f57a702724626b2c0a10ea052a6c2e1c8a697242ad14ab8"}
Sep 16 19:05:19.505902 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:19.505871    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7223-92ea32" event={"ID":"8660d141-86ac-4a78-befe-971a0a60e53b","Type":"ContainerStarted","Data":"322df3d522a384ca11740185c96e0050ac268f374209906c9685c10bc967b467"}
Sep 16 19:05:19.506027 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:19.505908    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7223-92ea32" event={"ID":"8660d141-86ac-4a78-befe-971a0a60e53b","Type":"ContainerStarted","Data":"71a55e2d59241e8af00025e6b9beee9a9be4a7834cb13d50028acb3b6d4a3374"}
Sep 16 19:05:19.506027 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:19.506011    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7223-c1b582" podUID="4a2c05c5-f132-44b8-826b-6547551ad7c8" containerName="test-curl-7223-c1b582" containerID="cri-o://19d836ecb512c97c4f3dabd6954105701ff03b440d895c75641ad2f6b84867ef" gracePeriod=2
Sep 16 19:05:19.546506 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:19.546454    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7223-92ea32" podStartSLOduration=1.546434515 podStartE2EDuration="1.546434515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:05:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:05:19.542974725 +0000 UTC m=+1547.330538863" watchObservedRunningTime="2026-09-16 19:05:19.546434515 +0000 UTC m=+1547.333998687"
Sep 16 19:05:19.786911 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:19.786839    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:05:19.787923 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:19.787895    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 16 19:05:19.788052 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:19.787944    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2" podUID="b46a4a50-fafe-4187-b140-220649dd54c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 16 19:05:20.149024 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:20.148945    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7223-92ea32"]
Sep 16 19:05:20.169614 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:20.166724    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7223-92ea32"]
Sep 16 19:05:20.515094 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:05:20.515013    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7223-92ea32" container="test-curl-7223-92ea32"
Sep 16 19:05:20.788080 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:20.787991    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 16 19:05:20.788383 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:20.788347    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2" podUID="b46a4a50-fafe-4187-b140-220649dd54c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 16 19:05:20.845195 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:20.844873    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn"]
Sep 16 19:05:20.867197 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:20.866600    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ec32e7-8brdn"]
Sep 16 19:05:21.080717 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.080612    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"]
Sep 16 19:05:21.081216 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.081192    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8660d141-86ac-4a78-befe-971a0a60e53b" containerName="test-curl-7223-92ea32"
Sep 16 19:05:21.081216 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.081214    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8660d141-86ac-4a78-befe-971a0a60e53b" containerName="test-curl-7223-92ea32"
Sep 16 19:05:21.081404 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.081338    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8660d141-86ac-4a78-befe-971a0a60e53b" containerName="test-curl-7223-92ea32"
Sep 16 19:05:21.083734 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.083711    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.093282 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.093260    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-d929c607-openshift-default-dockercfg-l84nr\""
Sep 16 19:05:21.126930 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.126882    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"]
Sep 16 19:05:21.286067 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.286033    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-workload-socket\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.286350 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.286095    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4x95\" (UniqueName: \"kubernetes.io/projected/2eb14464-0bdf-46f4-9ac3-9193779cfad5-kube-api-access-d4x95\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.286350 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.286186    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-data\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.286350 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.286280    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-podinfo\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.286350 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.286314    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-envoy\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.286350 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.286344    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-credential-socket\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.286649 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.286365    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-workload-certs\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.286649 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.286389    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-token\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.286649 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.286510    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istiod-ca-cert\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.387092 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.387053    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-workload-socket\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.387092 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.387091    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4x95\" (UniqueName: \"kubernetes.io/projected/2eb14464-0bdf-46f4-9ac3-9193779cfad5-kube-api-access-d4x95\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.387362 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.387114    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-data\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.387362 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.387281    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-podinfo\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.387362 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.387322    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-envoy\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.387362 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.387345    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-credential-socket\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.387362 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.387362    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-workload-certs\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.387615 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.387383    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-token\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.387615 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.387480    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istiod-ca-cert\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.387615 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.387542    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-workload-socket\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.387832 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.387812    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-data\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.387905 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.387874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-workload-certs\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.387954 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.387917    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-credential-socket\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.388251 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.388230    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istiod-ca-cert\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.389683 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.389653    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-envoy\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.389784 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.389739    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-podinfo\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.397611 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.397587    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4x95\" (UniqueName: \"kubernetes.io/projected/2eb14464-0bdf-46f4-9ac3-9193779cfad5-kube-api-access-d4x95\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.397704 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.397612    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-token\") pod \"e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") " pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.522082 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.522024    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7223-92ea32" podUID="8660d141-86ac-4a78-befe-971a0a60e53b" containerName="test-curl-7223-92ea32" containerID="cri-o://322df3d522a384ca11740185c96e0050ac268f374209906c9685c10bc967b467" gracePeriod=2
Sep 16 19:05:21.696433 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.696344    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:21.728374 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.728346    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7223-c1b582"
Sep 16 19:05:21.787868 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.787831    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 16 19:05:21.788010 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.787937    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2" podUID="b46a4a50-fafe-4187-b140-220649dd54c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 16 19:05:21.791862 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.791839    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7zxm\" (UniqueName: \"kubernetes.io/projected/4a2c05c5-f132-44b8-826b-6547551ad7c8-kube-api-access-v7zxm\") pod \"4a2c05c5-f132-44b8-826b-6547551ad7c8\" (UID: \"4a2c05c5-f132-44b8-826b-6547551ad7c8\") "
Sep 16 19:05:21.794828 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.794803    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a2c05c5-f132-44b8-826b-6547551ad7c8-kube-api-access-v7zxm" (OuterVolumeSpecName: "kube-api-access-v7zxm") pod "4a2c05c5-f132-44b8-826b-6547551ad7c8" (UID: "4a2c05c5-f132-44b8-826b-6547551ad7c8"). InnerVolumeSpecName "kube-api-access-v7zxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:21.893391 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:21.893355    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7zxm\" (UniqueName: \"kubernetes.io/projected/4a2c05c5-f132-44b8-826b-6547551ad7c8-kube-api-access-v7zxm\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:22.011375 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.011348    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"]
Sep 16 19:05:22.012316 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:05:22.012290    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eb14464_0bdf_46f4_9ac3_9193779cfad5.slice/crio-b1611d99276ee9f84d91a4a20c61b2b5876ab8500fe7bb8dd896ec612ebc58af WatchSource:0}: Error finding container b1611d99276ee9f84d91a4a20c61b2b5876ab8500fe7bb8dd896ec612ebc58af: Status 404 returned error can't find the container with id b1611d99276ee9f84d91a4a20c61b2b5876ab8500fe7bb8dd896ec612ebc58af
Sep 16 19:05:22.014614 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.014581    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:05:22.014691 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.014647    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:05:22.014691 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.014678    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:05:22.530068 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.530038    2578 generic.go:356] "Generic (PLEG): container finished" podID="4a2c05c5-f132-44b8-826b-6547551ad7c8" containerID="19d836ecb512c97c4f3dabd6954105701ff03b440d895c75641ad2f6b84867ef" exitCode=137
Sep 16 19:05:22.530564 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.530085    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7223-c1b582"
Sep 16 19:05:22.530564 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.530143    2578 scope.go:117] "RemoveContainer" containerID="19d836ecb512c97c4f3dabd6954105701ff03b440d895c75641ad2f6b84867ef"
Sep 16 19:05:22.532296 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.532266    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4" event={"ID":"2eb14464-0bdf-46f4-9ac3-9193779cfad5","Type":"ContainerStarted","Data":"b31afa33ddea5651910a58bbaa753efefbc4915c191d0b5075169990d05e5c86"}
Sep 16 19:05:22.532410 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.532327    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4" event={"ID":"2eb14464-0bdf-46f4-9ac3-9193779cfad5","Type":"ContainerStarted","Data":"b1611d99276ee9f84d91a4a20c61b2b5876ab8500fe7bb8dd896ec612ebc58af"}
Sep 16 19:05:22.548152 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.548126    2578 scope.go:117] "RemoveContainer" containerID="19d836ecb512c97c4f3dabd6954105701ff03b440d895c75641ad2f6b84867ef"
Sep 16 19:05:22.548834 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:05:22.548805    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19d836ecb512c97c4f3dabd6954105701ff03b440d895c75641ad2f6b84867ef\": container with ID starting with 19d836ecb512c97c4f3dabd6954105701ff03b440d895c75641ad2f6b84867ef not found: ID does not exist" containerID="19d836ecb512c97c4f3dabd6954105701ff03b440d895c75641ad2f6b84867ef"
Sep 16 19:05:22.549118 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.549089    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19d836ecb512c97c4f3dabd6954105701ff03b440d895c75641ad2f6b84867ef"} err="failed to get container status \"19d836ecb512c97c4f3dabd6954105701ff03b440d895c75641ad2f6b84867ef\": rpc error: code = NotFound desc = could not find container \"19d836ecb512c97c4f3dabd6954105701ff03b440d895c75641ad2f6b84867ef\": container with ID starting with 19d836ecb512c97c4f3dabd6954105701ff03b440d895c75641ad2f6b84867ef not found: ID does not exist"
Sep 16 19:05:22.568722 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:05:22.568676    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7223-c1b582\" is forbidden: User \"system:node:ip-10-0-134-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-234.ec2.internal' and this object" podUID="4a2c05c5-f132-44b8-826b-6547551ad7c8" pod="opendatahub/test-curl-7223-c1b582"
Sep 16 19:05:22.697143 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.696997    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:22.708002 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:05:22.707963    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7223-c1b582\" is forbidden: User \"system:node:ip-10-0-134-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-234.ec2.internal' and this object" podUID="4a2c05c5-f132-44b8-826b-6547551ad7c8" pod="opendatahub/test-curl-7223-c1b582"
Sep 16 19:05:22.710033 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.710007    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a2c05c5-f132-44b8-826b-6547551ad7c8" path="/var/lib/kubelet/pods/4a2c05c5-f132-44b8-826b-6547551ad7c8/volumes"
Sep 16 19:05:22.710393 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.710378    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e05e527f-b9b2-487e-bfee-a931d7b4843a" path="/var/lib/kubelet/pods/e05e527f-b9b2-487e-bfee-a931d7b4843a/volumes"
Sep 16 19:05:22.779841 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.779761    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4" podStartSLOduration=1.7797321940000002 podStartE2EDuration="1.779732194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:05:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:05:22.566501527 +0000 UTC m=+1550.354065701" watchObservedRunningTime="2026-09-16 19:05:22.779732194 +0000 UTC m=+1550.567296350"
Sep 16 19:05:22.781123 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.781097    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"]
Sep 16 19:05:22.786895 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.786863    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 16 19:05:22.787048 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:22.786908    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2" podUID="b46a4a50-fafe-4187-b140-220649dd54c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 16 19:05:23.145247 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:23.145206    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:05:23.702489 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:23.702461    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:23.769651 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:23.769620    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7223-92ea32"
Sep 16 19:05:23.787784 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:23.787711    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 16 19:05:23.787784 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:23.787760    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2" podUID="b46a4a50-fafe-4187-b140-220649dd54c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 16 19:05:23.814104 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:23.814069    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49vt4\" (UniqueName: \"kubernetes.io/projected/8660d141-86ac-4a78-befe-971a0a60e53b-kube-api-access-49vt4\") pod \"8660d141-86ac-4a78-befe-971a0a60e53b\" (UID: \"8660d141-86ac-4a78-befe-971a0a60e53b\") "
Sep 16 19:05:23.816246 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:23.816223    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8660d141-86ac-4a78-befe-971a0a60e53b-kube-api-access-49vt4" (OuterVolumeSpecName: "kube-api-access-49vt4") pod "8660d141-86ac-4a78-befe-971a0a60e53b" (UID: "8660d141-86ac-4a78-befe-971a0a60e53b"). InnerVolumeSpecName "kube-api-access-49vt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:23.914957 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:23.914917    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49vt4\" (UniqueName: \"kubernetes.io/projected/8660d141-86ac-4a78-befe-971a0a60e53b-kube-api-access-49vt4\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:24.555620 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:24.555457    2578 generic.go:356] "Generic (PLEG): container finished" podID="8660d141-86ac-4a78-befe-971a0a60e53b" containerID="322df3d522a384ca11740185c96e0050ac268f374209906c9685c10bc967b467" exitCode=137
Sep 16 19:05:24.555620 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:24.555504    2578 scope.go:117] "RemoveContainer" containerID="322df3d522a384ca11740185c96e0050ac268f374209906c9685c10bc967b467"
Sep 16 19:05:24.555998 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:24.555970    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7223-92ea32"
Sep 16 19:05:24.556284 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:24.556265    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:24.559831 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:24.559806    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:24.575886 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:24.575740    2578 scope.go:117] "RemoveContainer" containerID="322df3d522a384ca11740185c96e0050ac268f374209906c9685c10bc967b467"
Sep 16 19:05:24.576060 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:05:24.576040    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"322df3d522a384ca11740185c96e0050ac268f374209906c9685c10bc967b467\": container with ID starting with 322df3d522a384ca11740185c96e0050ac268f374209906c9685c10bc967b467 not found: ID does not exist" containerID="322df3d522a384ca11740185c96e0050ac268f374209906c9685c10bc967b467"
Sep 16 19:05:24.576106 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:24.576067    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"322df3d522a384ca11740185c96e0050ac268f374209906c9685c10bc967b467"} err="failed to get container status \"322df3d522a384ca11740185c96e0050ac268f374209906c9685c10bc967b467\": rpc error: code = NotFound desc = could not find container \"322df3d522a384ca11740185c96e0050ac268f374209906c9685c10bc967b467\": container with ID starting with 322df3d522a384ca11740185c96e0050ac268f374209906c9685c10bc967b467 not found: ID does not exist"
Sep 16 19:05:24.602444 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:05:24.602394    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7223-92ea32\" is forbidden: User \"system:node:ip-10-0-134-234.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-234.ec2.internal' and this object" podUID="8660d141-86ac-4a78-befe-971a0a60e53b" pod="opendatahub/test-curl-7223-92ea32"
Sep 16 19:05:24.712137 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:24.712103    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8660d141-86ac-4a78-befe-971a0a60e53b" path="/var/lib/kubelet/pods/8660d141-86ac-4a78-befe-971a0a60e53b/volumes"
Sep 16 19:05:24.786923 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:24.786885    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 16 19:05:24.787072 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:24.786953    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2" podUID="b46a4a50-fafe-4187-b140-220649dd54c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 16 19:05:25.791057 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:25.791024    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:05:25.792246 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:25.792223    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:05:30.169929 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:30.169897    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"]
Sep 16 19:05:30.170381 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:30.170123    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr" podUID="50e70c91-ac88-42eb-8d60-76e5e3869277" containerName="istio-proxy" containerID="cri-o://b4b27564f136806b37aeaf535e1634a0fc47ae812dd5a27aba9ab60ecdfbc85f" gracePeriod=30
Sep 16 19:05:30.548400 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:30.548317    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq"]
Sep 16 19:05:30.552084 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:30.552054    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq"
Sep 16 19:05:30.555027 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:30.555005    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nmr62\""
Sep 16 19:05:30.560678 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:30.560648    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq"]
Sep 16 19:05:30.585671 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:30.585641    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/800b0ccc-d7ee-478e-8f45-1492dd9d2381-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq\" (UID: \"800b0ccc-d7ee-478e-8f45-1492dd9d2381\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq"
Sep 16 19:05:30.687265 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:30.687225    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/800b0ccc-d7ee-478e-8f45-1492dd9d2381-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq\" (UID: \"800b0ccc-d7ee-478e-8f45-1492dd9d2381\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq"
Sep 16 19:05:30.687854 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:30.687834    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/800b0ccc-d7ee-478e-8f45-1492dd9d2381-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq\" (UID: \"800b0ccc-d7ee-478e-8f45-1492dd9d2381\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq"
Sep 16 19:05:30.865354 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:30.865323    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq"
Sep 16 19:05:31.154015 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:05:31.153973    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod800b0ccc_d7ee_478e_8f45_1492dd9d2381.slice/crio-55cb031995574bc829e5823f1d932bf4176d054d6a3e1ca0bd6cafe1412fd2b4 WatchSource:0}: Error finding container 55cb031995574bc829e5823f1d932bf4176d054d6a3e1ca0bd6cafe1412fd2b4: Status 404 returned error can't find the container with id 55cb031995574bc829e5823f1d932bf4176d054d6a3e1ca0bd6cafe1412fd2b4
Sep 16 19:05:31.158209 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:31.158152    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq"]
Sep 16 19:05:31.608016 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:31.607980    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq" event={"ID":"800b0ccc-d7ee-478e-8f45-1492dd9d2381","Type":"ContainerStarted","Data":"b5e5e73875ddfe40c77a264043d689cb019b80bba4dee24a72f4bcf79259c141"}
Sep 16 19:05:31.608534 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:31.608029    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq" event={"ID":"800b0ccc-d7ee-478e-8f45-1492dd9d2381","Type":"ContainerStarted","Data":"55cb031995574bc829e5823f1d932bf4176d054d6a3e1ca0bd6cafe1412fd2b4"}
Sep 16 19:05:31.626842 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:31.626794    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq" podStartSLOduration=1.626778169 podStartE2EDuration="1.626778169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:05:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:05:31.625235953 +0000 UTC m=+1559.412800130" watchObservedRunningTime="2026-09-16 19:05:31.626778169 +0000 UTC m=+1559.414342323"
Sep 16 19:05:32.616476 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:32.616445    2578 generic.go:356] "Generic (PLEG): container finished" podID="800b0ccc-d7ee-478e-8f45-1492dd9d2381" containerID="b5e5e73875ddfe40c77a264043d689cb019b80bba4dee24a72f4bcf79259c141" exitCode=0
Sep 16 19:05:32.616849 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:32.616529    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq" event={"ID":"800b0ccc-d7ee-478e-8f45-1492dd9d2381","Type":"ContainerDied","Data":"b5e5e73875ddfe40c77a264043d689cb019b80bba4dee24a72f4bcf79259c141"}
Sep 16 19:05:32.855547 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:32.855519    2578 scope.go:117] "RemoveContainer" containerID="d1d22f64340ae9da006729da237b57d033aea8895ef1a290898495b10c9fedba"
Sep 16 19:05:32.865408 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:32.865382    2578 scope.go:117] "RemoveContainer" containerID="64297f4f79e753c24d1366743680c275375c2e4566ad68c7a12879e1600a5fec"
Sep 16 19:05:33.767539 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:33.767513    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq"
Sep 16 19:05:33.819543 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:33.819509    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/800b0ccc-d7ee-478e-8f45-1492dd9d2381-maas-api-service-ca\") pod \"800b0ccc-d7ee-478e-8f45-1492dd9d2381\" (UID: \"800b0ccc-d7ee-478e-8f45-1492dd9d2381\") "
Sep 16 19:05:33.819902 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:33.819877    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/800b0ccc-d7ee-478e-8f45-1492dd9d2381-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "800b0ccc-d7ee-478e-8f45-1492dd9d2381" (UID: "800b0ccc-d7ee-478e-8f45-1492dd9d2381"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:05:33.921486 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:33.921395    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/800b0ccc-d7ee-478e-8f45-1492dd9d2381-maas-api-service-ca\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:34.631474 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:34.631445    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq"
Sep 16 19:05:34.631724 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:34.631489    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq" event={"ID":"800b0ccc-d7ee-478e-8f45-1492dd9d2381","Type":"ContainerDied","Data":"55cb031995574bc829e5823f1d932bf4176d054d6a3e1ca0bd6cafe1412fd2b4"}
Sep 16 19:05:34.631724 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:34.631581    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55cb031995574bc829e5823f1d932bf4176d054d6a3e1ca0bd6cafe1412fd2b4"
Sep 16 19:05:35.431389 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.431331    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 19:05:35.540301 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.540263    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-envoy\") pod \"50e70c91-ac88-42eb-8d60-76e5e3869277\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") "
Sep 16 19:05:35.540488 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.540311    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-podinfo\") pod \"50e70c91-ac88-42eb-8d60-76e5e3869277\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") "
Sep 16 19:05:35.540488 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.540343    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-workload-socket\") pod \"50e70c91-ac88-42eb-8d60-76e5e3869277\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") "
Sep 16 19:05:35.540488 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.540371    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50e70c91-ac88-42eb-8d60-76e5e3869277-istiod-ca-cert\") pod \"50e70c91-ac88-42eb-8d60-76e5e3869277\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") "
Sep 16 19:05:35.540488 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.540402    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-token\") pod \"50e70c91-ac88-42eb-8d60-76e5e3869277\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") "
Sep 16 19:05:35.540856 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.540521    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-credential-socket\") pod \"50e70c91-ac88-42eb-8d60-76e5e3869277\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") "
Sep 16 19:05:35.540856 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.540583    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6pl5\" (UniqueName: \"kubernetes.io/projected/50e70c91-ac88-42eb-8d60-76e5e3869277-kube-api-access-m6pl5\") pod \"50e70c91-ac88-42eb-8d60-76e5e3869277\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") "
Sep 16 19:05:35.540856 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.540623    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-data\") pod \"50e70c91-ac88-42eb-8d60-76e5e3869277\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") "
Sep 16 19:05:35.540856 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.540661    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-workload-certs\") pod \"50e70c91-ac88-42eb-8d60-76e5e3869277\" (UID: \"50e70c91-ac88-42eb-8d60-76e5e3869277\") "
Sep 16 19:05:35.540856 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.540747    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "50e70c91-ac88-42eb-8d60-76e5e3869277" (UID: "50e70c91-ac88-42eb-8d60-76e5e3869277"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:35.541107 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.540974    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "50e70c91-ac88-42eb-8d60-76e5e3869277" (UID: "50e70c91-ac88-42eb-8d60-76e5e3869277"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:35.541107 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.541074    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50e70c91-ac88-42eb-8d60-76e5e3869277-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "50e70c91-ac88-42eb-8d60-76e5e3869277" (UID: "50e70c91-ac88-42eb-8d60-76e5e3869277"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:05:35.541238 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.541151    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:35.541238 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.541184    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:35.541238 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.541215    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "50e70c91-ac88-42eb-8d60-76e5e3869277" (UID: "50e70c91-ac88-42eb-8d60-76e5e3869277"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:35.541352 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.541129    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-data" (OuterVolumeSpecName: "istio-data") pod "50e70c91-ac88-42eb-8d60-76e5e3869277" (UID: "50e70c91-ac88-42eb-8d60-76e5e3869277"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:35.543085 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.543058    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "50e70c91-ac88-42eb-8d60-76e5e3869277" (UID: "50e70c91-ac88-42eb-8d60-76e5e3869277"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:05:35.543259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.543242    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-token" (OuterVolumeSpecName: "istio-token") pod "50e70c91-ac88-42eb-8d60-76e5e3869277" (UID: "50e70c91-ac88-42eb-8d60-76e5e3869277"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:35.543331 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.543262    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50e70c91-ac88-42eb-8d60-76e5e3869277-kube-api-access-m6pl5" (OuterVolumeSpecName: "kube-api-access-m6pl5") pod "50e70c91-ac88-42eb-8d60-76e5e3869277" (UID: "50e70c91-ac88-42eb-8d60-76e5e3869277"). InnerVolumeSpecName "kube-api-access-m6pl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:35.543883 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.543864    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "50e70c91-ac88-42eb-8d60-76e5e3869277" (UID: "50e70c91-ac88-42eb-8d60-76e5e3869277"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:35.640265 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.640228    2578 generic.go:356] "Generic (PLEG): container finished" podID="50e70c91-ac88-42eb-8d60-76e5e3869277" containerID="b4b27564f136806b37aeaf535e1634a0fc47ae812dd5a27aba9ab60ecdfbc85f" exitCode=0
Sep 16 19:05:35.640471 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.640297    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"
Sep 16 19:05:35.640471 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.640331    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr" event={"ID":"50e70c91-ac88-42eb-8d60-76e5e3869277","Type":"ContainerDied","Data":"b4b27564f136806b37aeaf535e1634a0fc47ae812dd5a27aba9ab60ecdfbc85f"}
Sep 16 19:05:35.640471 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.640399    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr" event={"ID":"50e70c91-ac88-42eb-8d60-76e5e3869277","Type":"ContainerDied","Data":"16be00fe9dc61cb03055154b4e8c07bd1c67ee8429e096ea4306a4c4e9d45267"}
Sep 16 19:05:35.640471 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.640428    2578 scope.go:117] "RemoveContainer" containerID="b4b27564f136806b37aeaf535e1634a0fc47ae812dd5a27aba9ab60ecdfbc85f"
Sep 16 19:05:35.641969 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.641950    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50e70c91-ac88-42eb-8d60-76e5e3869277-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:35.641969 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.641968    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:35.641969 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.641978    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6pl5\" (UniqueName: \"kubernetes.io/projected/50e70c91-ac88-42eb-8d60-76e5e3869277-kube-api-access-m6pl5\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:35.642224 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.641994    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:35.642224 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.642006    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:35.642224 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.642013    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:35.642224 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.642022    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50e70c91-ac88-42eb-8d60-76e5e3869277-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:35.651519 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.651500    2578 scope.go:117] "RemoveContainer" containerID="2fa5e088ae00698d0f12046c880d48237e214b89b1f9987e347dcaa88b2448f7"
Sep 16 19:05:35.661216 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.661200    2578 scope.go:117] "RemoveContainer" containerID="b4b27564f136806b37aeaf535e1634a0fc47ae812dd5a27aba9ab60ecdfbc85f"
Sep 16 19:05:35.661504 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:05:35.661483    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4b27564f136806b37aeaf535e1634a0fc47ae812dd5a27aba9ab60ecdfbc85f\": container with ID starting with b4b27564f136806b37aeaf535e1634a0fc47ae812dd5a27aba9ab60ecdfbc85f not found: ID does not exist" containerID="b4b27564f136806b37aeaf535e1634a0fc47ae812dd5a27aba9ab60ecdfbc85f"
Sep 16 19:05:35.661561 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.661513    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4b27564f136806b37aeaf535e1634a0fc47ae812dd5a27aba9ab60ecdfbc85f"} err="failed to get container status \"b4b27564f136806b37aeaf535e1634a0fc47ae812dd5a27aba9ab60ecdfbc85f\": rpc error: code = NotFound desc = could not find container \"b4b27564f136806b37aeaf535e1634a0fc47ae812dd5a27aba9ab60ecdfbc85f\": container with ID starting with b4b27564f136806b37aeaf535e1634a0fc47ae812dd5a27aba9ab60ecdfbc85f not found: ID does not exist"
Sep 16 19:05:35.661561 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.661532    2578 scope.go:117] "RemoveContainer" containerID="2fa5e088ae00698d0f12046c880d48237e214b89b1f9987e347dcaa88b2448f7"
Sep 16 19:05:35.661797 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:05:35.661778    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fa5e088ae00698d0f12046c880d48237e214b89b1f9987e347dcaa88b2448f7\": container with ID starting with 2fa5e088ae00698d0f12046c880d48237e214b89b1f9987e347dcaa88b2448f7 not found: ID does not exist" containerID="2fa5e088ae00698d0f12046c880d48237e214b89b1f9987e347dcaa88b2448f7"
Sep 16 19:05:35.661852 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.661806    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fa5e088ae00698d0f12046c880d48237e214b89b1f9987e347dcaa88b2448f7"} err="failed to get container status \"2fa5e088ae00698d0f12046c880d48237e214b89b1f9987e347dcaa88b2448f7\": rpc error: code = NotFound desc = could not find container \"2fa5e088ae00698d0f12046c880d48237e214b89b1f9987e347dcaa88b2448f7\": container with ID starting with 2fa5e088ae00698d0f12046c880d48237e214b89b1f9987e347dcaa88b2448f7 not found: ID does not exist"
Sep 16 19:05:35.668319 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.668289    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"]
Sep 16 19:05:35.673355 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:35.673334    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-ec32e7-openshift-default-6f854ddbd8-vn7pr"]
Sep 16 19:05:36.710140 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.710108    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50e70c91-ac88-42eb-8d60-76e5e3869277" path="/var/lib/kubelet/pods/50e70c91-ac88-42eb-8d60-76e5e3869277/volumes"
Sep 16 19:05:36.876667 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.876632    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"]
Sep 16 19:05:36.877079 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.877067    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="800b0ccc-d7ee-478e-8f45-1492dd9d2381" containerName="revoke-keys"
Sep 16 19:05:36.877129 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.877081    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="800b0ccc-d7ee-478e-8f45-1492dd9d2381" containerName="revoke-keys"
Sep 16 19:05:36.877129 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.877089    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50e70c91-ac88-42eb-8d60-76e5e3869277" containerName="istio-proxy"
Sep 16 19:05:36.877129 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.877096    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="50e70c91-ac88-42eb-8d60-76e5e3869277" containerName="istio-proxy"
Sep 16 19:05:36.877129 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.877109    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50e70c91-ac88-42eb-8d60-76e5e3869277" containerName="istio-proxy"
Sep 16 19:05:36.877129 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.877115    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="50e70c91-ac88-42eb-8d60-76e5e3869277" containerName="istio-proxy"
Sep 16 19:05:36.877300 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.877220    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="50e70c91-ac88-42eb-8d60-76e5e3869277" containerName="istio-proxy"
Sep 16 19:05:36.877300 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.877231    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="800b0ccc-d7ee-478e-8f45-1492dd9d2381" containerName="revoke-keys"
Sep 16 19:05:36.880282 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.880265    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:36.884107 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.884084    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-c090c8\"/\"openshift-service-ca.crt\""
Sep 16 19:05:36.884246 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.884084    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-c090c8\"/\"default-dockercfg-9h2xx\""
Sep 16 19:05:36.884246 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.884186    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-c090c8\"/\"kube-root-ca.crt\""
Sep 16 19:05:36.884246 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.884186    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-c090c8\"/\"e2e-conflict-b-05590707-kserve-self-signed-certs\""
Sep 16 19:05:36.897062 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.897039    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"]
Sep 16 19:05:36.955067 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.955032    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-tls-certs\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:36.955289 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.955219    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-tmp-dir\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:36.955353 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.955304    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-dshm\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:36.955401 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.955380    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-home\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:36.955444 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.955417    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wkqn\" (UniqueName: \"kubernetes.io/projected/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-kube-api-access-8wkqn\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:36.955498 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:36.955461    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-model-cache\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:37.057019 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.056910    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-tmp-dir\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:37.057019 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.056986    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-dshm\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:37.057325 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.057030    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-home\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:37.057325 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.057058    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wkqn\" (UniqueName: \"kubernetes.io/projected/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-kube-api-access-8wkqn\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:37.057325 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.057099    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-model-cache\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:37.057325 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.057156    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-tls-certs\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:37.059108 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.058212    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-home\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:37.059108 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.058492    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-model-cache\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:37.059342 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.059206    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-tmp-dir\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:37.061289 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.061266    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-dshm\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:37.062265 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.061661    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-tls-certs\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:37.078022 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.077990    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wkqn\" (UniqueName: \"kubernetes.io/projected/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-kube-api-access-8wkqn\") pod \"e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") " pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:37.189149 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.189108    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:37.533201 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.532002    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"]
Sep 16 19:05:37.658351 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.658310    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z" event={"ID":"2eefc8c5-0b5a-4666-a4ec-1288f0712a70","Type":"ContainerStarted","Data":"9820f8395f7149a6ad9bece330c03436a557ea73e5bb38a86246ab0afe54d77e"}
Sep 16 19:05:37.658545 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.658363    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z" event={"ID":"2eefc8c5-0b5a-4666-a4ec-1288f0712a70","Type":"ContainerStarted","Data":"4d099c98fb04d0bbc71f37e48c15ba2724019c288987f4018ecd95b551a5ddfd"}
Sep 16 19:05:37.687311 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.687240    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z" podStartSLOduration=1.687217041 podStartE2EDuration="1.687217041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:05:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:05:37.682840093 +0000 UTC m=+1565.470404275" watchObservedRunningTime="2026-09-16 19:05:37.687217041 +0000 UTC m=+1565.474781197"
Sep 16 19:05:37.802712 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:37.802603    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw" podUID="bcff8da3-8c6c-461f-a622-1eb1f57c091f" containerName="main" containerID="cri-o://6b989acd46c5cec0b2e684ab292040bd52135fb59032f08f85bff1f4dbcd1ac1" gracePeriod=45
Sep 16 19:05:38.130540 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.130507    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:05:38.172196 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.170190    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcff8da3-8c6c-461f-a622-1eb1f57c091f-tls-certs\") pod \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") "
Sep 16 19:05:38.172196 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.170252    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ldrn\" (UniqueName: \"kubernetes.io/projected/bcff8da3-8c6c-461f-a622-1eb1f57c091f-kube-api-access-8ldrn\") pod \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") "
Sep 16 19:05:38.172196 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.170324    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-tmp-dir\") pod \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") "
Sep 16 19:05:38.172196 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.170465    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-home\") pod \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") "
Sep 16 19:05:38.172196 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.170494    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-model-cache\") pod \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") "
Sep 16 19:05:38.172196 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.170523    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-dshm\") pod \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\" (UID: \"bcff8da3-8c6c-461f-a622-1eb1f57c091f\") "
Sep 16 19:05:38.172196 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.171408    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bcff8da3-8c6c-461f-a622-1eb1f57c091f" (UID: "bcff8da3-8c6c-461f-a622-1eb1f57c091f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:38.172196 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.171935    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-model-cache" (OuterVolumeSpecName: "model-cache") pod "bcff8da3-8c6c-461f-a622-1eb1f57c091f" (UID: "bcff8da3-8c6c-461f-a622-1eb1f57c091f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:38.172196 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.172130    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-home" (OuterVolumeSpecName: "home") pod "bcff8da3-8c6c-461f-a622-1eb1f57c091f" (UID: "bcff8da3-8c6c-461f-a622-1eb1f57c091f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:38.175717 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.175566    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-dshm" (OuterVolumeSpecName: "dshm") pod "bcff8da3-8c6c-461f-a622-1eb1f57c091f" (UID: "bcff8da3-8c6c-461f-a622-1eb1f57c091f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:38.182208 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.181542    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcff8da3-8c6c-461f-a622-1eb1f57c091f-kube-api-access-8ldrn" (OuterVolumeSpecName: "kube-api-access-8ldrn") pod "bcff8da3-8c6c-461f-a622-1eb1f57c091f" (UID: "bcff8da3-8c6c-461f-a622-1eb1f57c091f"). InnerVolumeSpecName "kube-api-access-8ldrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:38.182208 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.181625    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcff8da3-8c6c-461f-a622-1eb1f57c091f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bcff8da3-8c6c-461f-a622-1eb1f57c091f" (UID: "bcff8da3-8c6c-461f-a622-1eb1f57c091f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:05:38.275208 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.271987    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-home\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:38.275208 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.272030    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-model-cache\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:38.275208 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.272045    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-dshm\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:38.275208 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.272059    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcff8da3-8c6c-461f-a622-1eb1f57c091f-tls-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:38.275208 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.272075    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ldrn\" (UniqueName: \"kubernetes.io/projected/bcff8da3-8c6c-461f-a622-1eb1f57c091f-kube-api-access-8ldrn\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:38.275208 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.272090    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcff8da3-8c6c-461f-a622-1eb1f57c091f-tmp-dir\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:38.671134 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.671092    2578 generic.go:356] "Generic (PLEG): container finished" podID="bcff8da3-8c6c-461f-a622-1eb1f57c091f" containerID="6b989acd46c5cec0b2e684ab292040bd52135fb59032f08f85bff1f4dbcd1ac1" exitCode=0
Sep 16 19:05:38.671970 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.671945    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"
Sep 16 19:05:38.672765 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.672544    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw" event={"ID":"bcff8da3-8c6c-461f-a622-1eb1f57c091f","Type":"ContainerDied","Data":"6b989acd46c5cec0b2e684ab292040bd52135fb59032f08f85bff1f4dbcd1ac1"}
Sep 16 19:05:38.672765 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.672596    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw" event={"ID":"bcff8da3-8c6c-461f-a622-1eb1f57c091f","Type":"ContainerDied","Data":"5da96c5be3385cebc112b23fca23a2857fe79330359b2354804e7873324079eb"}
Sep 16 19:05:38.672765 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.672633    2578 scope.go:117] "RemoveContainer" containerID="6b989acd46c5cec0b2e684ab292040bd52135fb59032f08f85bff1f4dbcd1ac1"
Sep 16 19:05:38.683164 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.683146    2578 scope.go:117] "RemoveContainer" containerID="6b989acd46c5cec0b2e684ab292040bd52135fb59032f08f85bff1f4dbcd1ac1"
Sep 16 19:05:38.683492 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:05:38.683469    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b989acd46c5cec0b2e684ab292040bd52135fb59032f08f85bff1f4dbcd1ac1\": container with ID starting with 6b989acd46c5cec0b2e684ab292040bd52135fb59032f08f85bff1f4dbcd1ac1 not found: ID does not exist" containerID="6b989acd46c5cec0b2e684ab292040bd52135fb59032f08f85bff1f4dbcd1ac1"
Sep 16 19:05:38.683535 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.683500    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b989acd46c5cec0b2e684ab292040bd52135fb59032f08f85bff1f4dbcd1ac1"} err="failed to get container status \"6b989acd46c5cec0b2e684ab292040bd52135fb59032f08f85bff1f4dbcd1ac1\": rpc error: code = NotFound desc = could not find container \"6b989acd46c5cec0b2e684ab292040bd52135fb59032f08f85bff1f4dbcd1ac1\": container with ID starting with 6b989acd46c5cec0b2e684ab292040bd52135fb59032f08f85bff1f4dbcd1ac1 not found: ID does not exist"
Sep 16 19:05:38.699867 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.699837    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"]
Sep 16 19:05:38.710003 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:38.709980    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-8a45a9/test-model-8a45a9-kserve-679db5c4b9-q28rw"]
Sep 16 19:05:40.710421 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:40.710386    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcff8da3-8c6c-461f-a622-1eb1f57c091f" path="/var/lib/kubelet/pods/bcff8da3-8c6c-461f-a622-1eb1f57c091f/volumes"
Sep 16 19:05:41.405188 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:41.405151    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a45a9-29826420-2k9sr"]
Sep 16 19:05:41.409426 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:41.409399    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a45a9-29826420-2k9sr"]
Sep 16 19:05:41.646191 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:41.646139    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq"]
Sep 16 19:05:41.654094 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:41.654062    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a45a9-mfbpq"]
Sep 16 19:05:42.709538 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:42.709505    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="800b0ccc-d7ee-478e-8f45-1492dd9d2381" path="/var/lib/kubelet/pods/800b0ccc-d7ee-478e-8f45-1492dd9d2381/volumes"
Sep 16 19:05:42.709901 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:42.709827    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6fc2bbf-a9f4-465b-9c81-acc8340e3111" path="/var/lib/kubelet/pods/d6fc2bbf-a9f4-465b-9c81-acc8340e3111/volumes"
Sep 16 19:05:46.729661 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:46.729554    2578 generic.go:356] "Generic (PLEG): container finished" podID="2eb14464-0bdf-46f4-9ac3-9193779cfad5" containerID="b31afa33ddea5651910a58bbaa753efefbc4915c191d0b5075169990d05e5c86" exitCode=0
Sep 16 19:05:46.729661 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:46.729598    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4" event={"ID":"2eb14464-0bdf-46f4-9ac3-9193779cfad5","Type":"ContainerDied","Data":"b31afa33ddea5651910a58bbaa753efefbc4915c191d0b5075169990d05e5c86"}
Sep 16 19:05:46.730244 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:46.730104    2578 scope.go:117] "RemoveContainer" containerID="b31afa33ddea5651910a58bbaa753efefbc4915c191d0b5075169990d05e5c86"
Sep 16 19:05:46.731419 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:46.731385    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:05:46.731523 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:46.731463    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:05:46.731523 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:46.731502    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:05:47.190267 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:47.190218    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:47.190522 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:47.190297    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:47.207720 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:47.207688    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:47.739061 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:47.739012    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4" event={"ID":"2eb14464-0bdf-46f4-9ac3-9193779cfad5","Type":"ContainerStarted","Data":"1155217ee189e821cb1e5965da9f1b9be831ada7af4bc467578caecb5392ad41"}
Sep 16 19:05:47.775501 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:47.775469    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:05:48.697144 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:48.697104    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:48.697338 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:48.697282    2578 patch_prober.go:28] interesting pod/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 16 19:05:48.697397 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:48.697352    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4" podUID="2eb14464-0bdf-46f4-9ac3-9193779cfad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 16 19:05:49.697732 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:49.697696    2578 patch_prober.go:28] interesting pod/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 16 19:05:49.698080 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:49.697777    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4" podUID="2eb14464-0bdf-46f4-9ac3-9193779cfad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 16 19:05:50.576024 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:50.575986    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"]
Sep 16 19:05:50.576579 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:50.576542    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm" podUID="7e716728-4c89-4ce6-a53d-e9d8cc7a0731" containerName="istio-proxy" containerID="cri-o://c67d850975468a65e84e38176a5086a98c9e8597b123f68a06b935792aae81b1" gracePeriod=30
Sep 16 19:05:50.697045 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:50.697010    2578 patch_prober.go:28] interesting pod/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 16 19:05:50.697496 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:50.697451    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4" podUID="2eb14464-0bdf-46f4-9ac3-9193779cfad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 16 19:05:51.479825 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.479794    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"]
Sep 16 19:05:51.480345 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.480326    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcff8da3-8c6c-461f-a622-1eb1f57c091f" containerName="main"
Sep 16 19:05:51.480428 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.480347    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcff8da3-8c6c-461f-a622-1eb1f57c091f" containerName="main"
Sep 16 19:05:51.480482 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.480449    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcff8da3-8c6c-461f-a622-1eb1f57c091f" containerName="main"
Sep 16 19:05:51.480482 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.480462    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="50e70c91-ac88-42eb-8d60-76e5e3869277" containerName="istio-proxy"
Sep 16 19:05:51.487671 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.487651    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.496155 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.496131    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-ca3f9a-openshift-default-dockercfg-rkqgl\""
Sep 16 19:05:51.505073 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.505047    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"]
Sep 16 19:05:51.623858 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.623815    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a223baca-28f9-49d2-bd57-798b4350d102-istiod-ca-cert\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.624673 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.624121    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-workload-certs\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.624673 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.624251    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f8zc\" (UniqueName: \"kubernetes.io/projected/a223baca-28f9-49d2-bd57-798b4350d102-kube-api-access-4f8zc\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.624673 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.624281    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a223baca-28f9-49d2-bd57-798b4350d102-istio-podinfo\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.624673 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.624351    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a223baca-28f9-49d2-bd57-798b4350d102-istio-token\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.624673 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.624394    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-workload-socket\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.624673 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.624440    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-istio-envoy\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.624673 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.624493    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-credential-socket\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.624673 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.624537    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-istio-data\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.697321 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.697280    2578 patch_prober.go:28] interesting pod/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 16 19:05:51.697509 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.697365    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4" podUID="2eb14464-0bdf-46f4-9ac3-9193779cfad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 16 19:05:51.725225 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.725188    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a223baca-28f9-49d2-bd57-798b4350d102-istio-token\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.725383 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.725249    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-workload-socket\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.725383 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.725293    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-istio-envoy\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.725383 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.725339    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-credential-socket\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.725498 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.725383    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-istio-data\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.725498 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.725435    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a223baca-28f9-49d2-bd57-798b4350d102-istiod-ca-cert\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.725498 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.725465    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-workload-certs\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.725613 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.725545    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f8zc\" (UniqueName: \"kubernetes.io/projected/a223baca-28f9-49d2-bd57-798b4350d102-kube-api-access-4f8zc\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.725613 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.725570    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a223baca-28f9-49d2-bd57-798b4350d102-istio-podinfo\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.725835 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.725807    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-credential-socket\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.725968 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.725945    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-workload-socket\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.726103 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.726078    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-workload-certs\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.726355 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.726330    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-istio-data\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.726437 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.726343    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a223baca-28f9-49d2-bd57-798b4350d102-istiod-ca-cert\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.728002 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.727983    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-istio-envoy\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.728597 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.728576    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a223baca-28f9-49d2-bd57-798b4350d102-istio-podinfo\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.736130 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.736077    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a223baca-28f9-49d2-bd57-798b4350d102-istio-token\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.736465 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.736445    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f8zc\" (UniqueName: \"kubernetes.io/projected/a223baca-28f9-49d2-bd57-798b4350d102-kube-api-access-4f8zc\") pod \"e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") " pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:51.799804 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:51.799769    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:52.117750 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:52.117720    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"]
Sep 16 19:05:52.121010 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:05:52.120966    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda223baca_28f9_49d2_bd57_798b4350d102.slice/crio-61caea3597b3e24432c443e44a40c2565ac208ce9b46ab212a54c9bdb75be9ee WatchSource:0}: Error finding container 61caea3597b3e24432c443e44a40c2565ac208ce9b46ab212a54c9bdb75be9ee: Status 404 returned error can't find the container with id 61caea3597b3e24432c443e44a40c2565ac208ce9b46ab212a54c9bdb75be9ee
Sep 16 19:05:52.123574 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:52.123485    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:05:52.123574 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:52.123550    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:05:52.123705 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:52.123579    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:05:52.697153 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:52.697116    2578 patch_prober.go:28] interesting pod/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Sep 16 19:05:52.697568 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:52.697185    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4" podUID="2eb14464-0bdf-46f4-9ac3-9193779cfad5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Sep 16 19:05:52.786140 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:52.786099    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r" event={"ID":"a223baca-28f9-49d2-bd57-798b4350d102","Type":"ContainerStarted","Data":"de976220f70c568f2cb05275fb6abf599190a1dafdbe789c248b07cfb66c9777"}
Sep 16 19:05:52.786355 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:52.786158    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r" event={"ID":"a223baca-28f9-49d2-bd57-798b4350d102","Type":"ContainerStarted","Data":"61caea3597b3e24432c443e44a40c2565ac208ce9b46ab212a54c9bdb75be9ee"}
Sep 16 19:05:52.800486 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:52.800454    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:52.820130 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:52.820081    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r" podStartSLOduration=1.820064143 podStartE2EDuration="1.820064143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:05:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:05:52.816040189 +0000 UTC m=+1580.603604361" watchObservedRunningTime="2026-09-16 19:05:52.820064143 +0000 UTC m=+1580.607628297"
Sep 16 19:05:53.700993 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:53.700961    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:53.701402 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:53.701273    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:53.701910 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:53.701893    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:05:53.806156 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:53.806128    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:54.801472 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:54.801436    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:54.802522 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:54.802503    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:05:55.812530 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.812499    2578 generic.go:356] "Generic (PLEG): container finished" podID="7e716728-4c89-4ce6-a53d-e9d8cc7a0731" containerID="c67d850975468a65e84e38176a5086a98c9e8597b123f68a06b935792aae81b1" exitCode=0
Sep 16 19:05:55.812872 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.812577    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm" event={"ID":"7e716728-4c89-4ce6-a53d-e9d8cc7a0731","Type":"ContainerDied","Data":"c67d850975468a65e84e38176a5086a98c9e8597b123f68a06b935792aae81b1"}
Sep 16 19:05:55.812872 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.812644    2578 scope.go:117] "RemoveContainer" containerID="96078b8777467eb5c4d0330d02fc9a2d310ac02431160bf1bff2d6e84985aa35"
Sep 16 19:05:55.839408 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.839341    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 19:05:55.977790 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.977752    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-envoy\") pod \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") "
Sep 16 19:05:55.977993 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.977807    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-credential-socket\") pod \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") "
Sep 16 19:05:55.977993 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.977852    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-workload-socket\") pod \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") "
Sep 16 19:05:55.977993 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.977944    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6xkx\" (UniqueName: \"kubernetes.io/projected/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-kube-api-access-x6xkx\") pod \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") "
Sep 16 19:05:55.977993 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.977983    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-token\") pod \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") "
Sep 16 19:05:55.978277 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.978009    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-workload-certs\") pod \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") "
Sep 16 19:05:55.978277 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.978082    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-podinfo\") pod \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") "
Sep 16 19:05:55.978277 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.978149    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-data\") pod \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") "
Sep 16 19:05:55.978277 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.978221    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istiod-ca-cert\") pod \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\" (UID: \"7e716728-4c89-4ce6-a53d-e9d8cc7a0731\") "
Sep 16 19:05:55.982061 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.982023    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7e716728-4c89-4ce6-a53d-e9d8cc7a0731" (UID: "7e716728-4c89-4ce6-a53d-e9d8cc7a0731"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:55.982599 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.982559    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7e716728-4c89-4ce6-a53d-e9d8cc7a0731" (UID: "7e716728-4c89-4ce6-a53d-e9d8cc7a0731"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:55.984210 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.984167    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7e716728-4c89-4ce6-a53d-e9d8cc7a0731" (UID: "7e716728-4c89-4ce6-a53d-e9d8cc7a0731"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:55.987340 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.986603    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7e716728-4c89-4ce6-a53d-e9d8cc7a0731" (UID: "7e716728-4c89-4ce6-a53d-e9d8cc7a0731"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:55.987340 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.986697    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-data" (OuterVolumeSpecName: "istio-data") pod "7e716728-4c89-4ce6-a53d-e9d8cc7a0731" (UID: "7e716728-4c89-4ce6-a53d-e9d8cc7a0731"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:55.987340 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.986786    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-kube-api-access-x6xkx" (OuterVolumeSpecName: "kube-api-access-x6xkx") pod "7e716728-4c89-4ce6-a53d-e9d8cc7a0731" (UID: "7e716728-4c89-4ce6-a53d-e9d8cc7a0731"). InnerVolumeSpecName "kube-api-access-x6xkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:55.987781 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.987755    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7e716728-4c89-4ce6-a53d-e9d8cc7a0731" (UID: "7e716728-4c89-4ce6-a53d-e9d8cc7a0731"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:05:55.988946 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.988920    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7e716728-4c89-4ce6-a53d-e9d8cc7a0731" (UID: "7e716728-4c89-4ce6-a53d-e9d8cc7a0731"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:05:55.989674 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:55.989652    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-token" (OuterVolumeSpecName: "istio-token") pod "7e716728-4c89-4ce6-a53d-e9d8cc7a0731" (UID: "7e716728-4c89-4ce6-a53d-e9d8cc7a0731"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:56.080315 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:56.080283    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:56.080315 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:56.080310    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:56.080315 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:56.080319    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:56.080533 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:56.080327    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:56.080533 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:56.080335    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:56.080533 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:56.080344    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:56.080533 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:56.080353    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6xkx\" (UniqueName: \"kubernetes.io/projected/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-kube-api-access-x6xkx\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:56.080533 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:56.080361    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:56.080533 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:56.080369    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e716728-4c89-4ce6-a53d-e9d8cc7a0731-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:05:56.820697 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:56.820663    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"
Sep 16 19:05:56.821154 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:56.820694    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm" event={"ID":"7e716728-4c89-4ce6-a53d-e9d8cc7a0731","Type":"ContainerDied","Data":"0b84f988da673b39272a970101c5f81a5dade060a9692feb0f365a9ee46f2cb4"}
Sep 16 19:05:56.821154 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:56.820752    2578 scope.go:117] "RemoveContainer" containerID="c67d850975468a65e84e38176a5086a98c9e8597b123f68a06b935792aae81b1"
Sep 16 19:05:56.867624 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:56.867591    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"]
Sep 16 19:05:56.872246 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:56.872221    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-8a45a9-openshift-default-644bb874b8-w5dvm"]
Sep 16 19:05:58.710338 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:05:58.710305    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e716728-4c89-4ce6-a53d-e9d8cc7a0731" path="/var/lib/kubelet/pods/7e716728-4c89-4ce6-a53d-e9d8cc7a0731/volumes"
Sep 16 19:06:25.414962 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.414883    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"]
Sep 16 19:06:25.415379 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.415367    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e716728-4c89-4ce6-a53d-e9d8cc7a0731" containerName="istio-proxy"
Sep 16 19:06:25.415423 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.415381    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e716728-4c89-4ce6-a53d-e9d8cc7a0731" containerName="istio-proxy"
Sep 16 19:06:25.415423 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.415394    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e716728-4c89-4ce6-a53d-e9d8cc7a0731" containerName="istio-proxy"
Sep 16 19:06:25.415423 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.415400    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e716728-4c89-4ce6-a53d-e9d8cc7a0731" containerName="istio-proxy"
Sep 16 19:06:25.415517 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.415466    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e716728-4c89-4ce6-a53d-e9d8cc7a0731" containerName="istio-proxy"
Sep 16 19:06:25.415517 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.415478    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e716728-4c89-4ce6-a53d-e9d8cc7a0731" containerName="istio-proxy"
Sep 16 19:06:25.419102 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.419083    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.422012 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.421990    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-2dc9fc-openshift-default-dockercfg-vjl2l\""
Sep 16 19:06:25.466703 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.466667    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"]
Sep 16 19:06:25.502931 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.502886    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-envoy\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.503412 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.503384    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-token\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.503579 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.503563    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-data\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.503780 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.503761    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istiod-ca-cert\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.503930 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.503914    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-workload-socket\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.504068 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.504054    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-workload-certs\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.504276 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.504255    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb4wk\" (UniqueName: \"kubernetes.io/projected/73c1758a-a532-4c93-9e2f-c2236f5d58b3-kube-api-access-vb4wk\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.504428 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.504412    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-podinfo\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.504587 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.504571    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-credential-socket\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.606203 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.606147    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istiod-ca-cert\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.606408 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.606258    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-workload-socket\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.606408 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.606320    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-workload-certs\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.606408 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.606380    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb4wk\" (UniqueName: \"kubernetes.io/projected/73c1758a-a532-4c93-9e2f-c2236f5d58b3-kube-api-access-vb4wk\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.606579 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.606430    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-podinfo\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.606579 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.606471    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-credential-socket\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.606685 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.606581    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-envoy\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.606685 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.606608    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-token\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.606685 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.606642    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-data\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.607046 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.607015    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istiod-ca-cert\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.607158 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.607032    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-workload-socket\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.607371 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.607340    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-credential-socket\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.607561 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.607525    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-workload-certs\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.607772 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.607747    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-data\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.609695 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.609657    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-envoy\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.610025 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.610006    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-podinfo\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.617874 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.617829    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-token\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.618167 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.618147    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb4wk\" (UniqueName: \"kubernetes.io/projected/73c1758a-a532-4c93-9e2f-c2236f5d58b3-kube-api-access-vb4wk\") pod \"e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") " pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:25.733247 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:25.733152    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:26.026334 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:26.026302    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"]
Sep 16 19:06:26.027224 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:06:26.027143    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c1758a_a532_4c93_9e2f_c2236f5d58b3.slice/crio-f209ba774e0c1522eecbbce1684a8bb0af71182af9573a8de5d23f4f5ef5170d WatchSource:0}: Error finding container f209ba774e0c1522eecbbce1684a8bb0af71182af9573a8de5d23f4f5ef5170d: Status 404 returned error can't find the container with id f209ba774e0c1522eecbbce1684a8bb0af71182af9573a8de5d23f4f5ef5170d
Sep 16 19:06:26.030148 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:26.030110    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:06:26.030257 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:26.030195    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:06:26.030257 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:26.030227    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:06:27.029236 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:27.029192    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql" event={"ID":"73c1758a-a532-4c93-9e2f-c2236f5d58b3","Type":"ContainerStarted","Data":"86943eaf572e680baa52987808a6ada3527fdfef92875660333c80e6507366c8"}
Sep 16 19:06:27.029640 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:27.029264    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql" event={"ID":"73c1758a-a532-4c93-9e2f-c2236f5d58b3","Type":"ContainerStarted","Data":"f209ba774e0c1522eecbbce1684a8bb0af71182af9573a8de5d23f4f5ef5170d"}
Sep 16 19:06:27.053906 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:27.053794    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql" podStartSLOduration=2.053774491 podStartE2EDuration="2.053774491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:06:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:06:27.050401718 +0000 UTC m=+1614.837965890" watchObservedRunningTime="2026-09-16 19:06:27.053774491 +0000 UTC m=+1614.841338644"
Sep 16 19:06:27.733612 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:27.733569    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:27.738905 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:27.738874    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:28.036340 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:28.036258    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:28.037488 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:28.037458    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:06:28.674653 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:28.674617    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"]
Sep 16 19:06:28.674827 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:28.674810    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg" podUID="5c1295f7-3e69-4025-9381-fa348f881123" containerName="istio-proxy" containerID="cri-o://f70ccdbcb4556c24c3d22797c2a26cab5abb46a7ebfd682623f629f099c7c26f" gracePeriod=30
Sep 16 19:06:30.051234 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:30.051196    2578 generic.go:356] "Generic (PLEG): container finished" podID="a223baca-28f9-49d2-bd57-798b4350d102" containerID="de976220f70c568f2cb05275fb6abf599190a1dafdbe789c248b07cfb66c9777" exitCode=0
Sep 16 19:06:30.051684 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:30.051269    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r" event={"ID":"a223baca-28f9-49d2-bd57-798b4350d102","Type":"ContainerDied","Data":"de976220f70c568f2cb05275fb6abf599190a1dafdbe789c248b07cfb66c9777"}
Sep 16 19:06:30.051842 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:30.051827    2578 scope.go:117] "RemoveContainer" containerID="de976220f70c568f2cb05275fb6abf599190a1dafdbe789c248b07cfb66c9777"
Sep 16 19:06:30.053064 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:30.053037    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:06:30.053129 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:30.053104    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:06:30.053164 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:30.053154    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:06:31.071003 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:31.070964    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r" event={"ID":"a223baca-28f9-49d2-bd57-798b4350d102","Type":"ContainerStarted","Data":"53a3adc80a31ea2115cb82e785bcf3a34cc8bf1f07e5edd330f55d8ce5ee48f4"}
Sep 16 19:06:31.800663 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:31.800625    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:06:31.800899 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:31.800877    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 16 19:06:31.800948 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:31.800923    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r" podUID="a223baca-28f9-49d2-bd57-798b4350d102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 16 19:06:32.800483 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:32.800450    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 16 19:06:32.800850 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:32.800503    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r" podUID="a223baca-28f9-49d2-bd57-798b4350d102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 16 19:06:32.916081 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:32.916048    2578 scope.go:117] "RemoveContainer" containerID="77a47b8d018d872b7424c615daf1aaf15ed56d143293022483e29d476ab4dd9a"
Sep 16 19:06:32.925497 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:32.925477    2578 scope.go:117] "RemoveContainer" containerID="8f4bf74569e9dec67bb8750f7e18a98f5177a557ca3c27b856422a15f096b951"
Sep 16 19:06:32.934381 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:32.934359    2578 scope.go:117] "RemoveContainer" containerID="fc779e6146893edf719a2e9ee6b29ec3e7dc3cec4c9d72afa76b257e147e5486"
Sep 16 19:06:32.964128 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:32.964102    2578 scope.go:117] "RemoveContainer" containerID="2a8c4d24098de8c570897e751042510937886704d0a90cb174fa4e5152cf8fe0"
Sep 16 19:06:33.800280 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:33.800246    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 16 19:06:33.800430 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:33.800302    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r" podUID="a223baca-28f9-49d2-bd57-798b4350d102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 16 19:06:33.977397 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:33.977373    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:06:34.003371 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.003340    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-credential-socket\") pod \"5c1295f7-3e69-4025-9381-fa348f881123\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") "
Sep 16 19:06:34.003546 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.003384    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c1295f7-3e69-4025-9381-fa348f881123-istio-podinfo\") pod \"5c1295f7-3e69-4025-9381-fa348f881123\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") "
Sep 16 19:06:34.003546 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.003407    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmg7l\" (UniqueName: \"kubernetes.io/projected/5c1295f7-3e69-4025-9381-fa348f881123-kube-api-access-mmg7l\") pod \"5c1295f7-3e69-4025-9381-fa348f881123\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") "
Sep 16 19:06:34.003546 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.003432    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-workload-certs\") pod \"5c1295f7-3e69-4025-9381-fa348f881123\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") "
Sep 16 19:06:34.003546 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.003469    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-istio-envoy\") pod \"5c1295f7-3e69-4025-9381-fa348f881123\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") "
Sep 16 19:06:34.003831 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.003551    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-istio-data\") pod \"5c1295f7-3e69-4025-9381-fa348f881123\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") "
Sep 16 19:06:34.003831 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.003594    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c1295f7-3e69-4025-9381-fa348f881123-istio-token\") pod \"5c1295f7-3e69-4025-9381-fa348f881123\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") "
Sep 16 19:06:34.003831 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.003652    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5c1295f7-3e69-4025-9381-fa348f881123" (UID: "5c1295f7-3e69-4025-9381-fa348f881123"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:06:34.003831 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.003684    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5c1295f7-3e69-4025-9381-fa348f881123" (UID: "5c1295f7-3e69-4025-9381-fa348f881123"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:06:34.003831 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.003694    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-workload-socket\") pod \"5c1295f7-3e69-4025-9381-fa348f881123\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") "
Sep 16 19:06:34.003831 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.003758    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c1295f7-3e69-4025-9381-fa348f881123-istiod-ca-cert\") pod \"5c1295f7-3e69-4025-9381-fa348f881123\" (UID: \"5c1295f7-3e69-4025-9381-fa348f881123\") "
Sep 16 19:06:34.004145 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.003873    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-istio-data" (OuterVolumeSpecName: "istio-data") pod "5c1295f7-3e69-4025-9381-fa348f881123" (UID: "5c1295f7-3e69-4025-9381-fa348f881123"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:06:34.004145 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.003995    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5c1295f7-3e69-4025-9381-fa348f881123" (UID: "5c1295f7-3e69-4025-9381-fa348f881123"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:06:34.004145 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.004140    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c1295f7-3e69-4025-9381-fa348f881123-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5c1295f7-3e69-4025-9381-fa348f881123" (UID: "5c1295f7-3e69-4025-9381-fa348f881123"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:06:34.004348 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.004267    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c1295f7-3e69-4025-9381-fa348f881123-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:06:34.004348 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.004286    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:06:34.004348 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.004300    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:06:34.004348 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.004312    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:06:34.004348 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.004319    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:06:34.005666 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.005639    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5c1295f7-3e69-4025-9381-fa348f881123" (UID: "5c1295f7-3e69-4025-9381-fa348f881123"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:06:34.005832 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.005808    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c1295f7-3e69-4025-9381-fa348f881123-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5c1295f7-3e69-4025-9381-fa348f881123" (UID: "5c1295f7-3e69-4025-9381-fa348f881123"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:06:34.006093 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.006072    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c1295f7-3e69-4025-9381-fa348f881123-istio-token" (OuterVolumeSpecName: "istio-token") pod "5c1295f7-3e69-4025-9381-fa348f881123" (UID: "5c1295f7-3e69-4025-9381-fa348f881123"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:06:34.006207 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.006105    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c1295f7-3e69-4025-9381-fa348f881123-kube-api-access-mmg7l" (OuterVolumeSpecName: "kube-api-access-mmg7l") pod "5c1295f7-3e69-4025-9381-fa348f881123" (UID: "5c1295f7-3e69-4025-9381-fa348f881123"). InnerVolumeSpecName "kube-api-access-mmg7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:06:34.098733 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.098696    2578 generic.go:356] "Generic (PLEG): container finished" podID="5c1295f7-3e69-4025-9381-fa348f881123" containerID="f70ccdbcb4556c24c3d22797c2a26cab5abb46a7ebfd682623f629f099c7c26f" exitCode=0
Sep 16 19:06:34.098892 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.098741    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"
Sep 16 19:06:34.098892 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.098781    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg" event={"ID":"5c1295f7-3e69-4025-9381-fa348f881123","Type":"ContainerDied","Data":"f70ccdbcb4556c24c3d22797c2a26cab5abb46a7ebfd682623f629f099c7c26f"}
Sep 16 19:06:34.098892 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.098863    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg" event={"ID":"5c1295f7-3e69-4025-9381-fa348f881123","Type":"ContainerDied","Data":"278364c89a6acbfe6157d185115eaefb4fa07d7331b226a3a71287e0ced303bb"}
Sep 16 19:06:34.099046 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.098894    2578 scope.go:117] "RemoveContainer" containerID="f70ccdbcb4556c24c3d22797c2a26cab5abb46a7ebfd682623f629f099c7c26f"
Sep 16 19:06:34.104969 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.104944    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c1295f7-3e69-4025-9381-fa348f881123-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:06:34.104969 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.104968    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmg7l\" (UniqueName: \"kubernetes.io/projected/5c1295f7-3e69-4025-9381-fa348f881123-kube-api-access-mmg7l\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:06:34.105131 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.104978    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c1295f7-3e69-4025-9381-fa348f881123-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:06:34.105131 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.104987    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c1295f7-3e69-4025-9381-fa348f881123-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:06:34.109861 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.109844    2578 scope.go:117] "RemoveContainer" containerID="f70ccdbcb4556c24c3d22797c2a26cab5abb46a7ebfd682623f629f099c7c26f"
Sep 16 19:06:34.110137 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:06:34.110119    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f70ccdbcb4556c24c3d22797c2a26cab5abb46a7ebfd682623f629f099c7c26f\": container with ID starting with f70ccdbcb4556c24c3d22797c2a26cab5abb46a7ebfd682623f629f099c7c26f not found: ID does not exist" containerID="f70ccdbcb4556c24c3d22797c2a26cab5abb46a7ebfd682623f629f099c7c26f"
Sep 16 19:06:34.110225 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.110144    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f70ccdbcb4556c24c3d22797c2a26cab5abb46a7ebfd682623f629f099c7c26f"} err="failed to get container status \"f70ccdbcb4556c24c3d22797c2a26cab5abb46a7ebfd682623f629f099c7c26f\": rpc error: code = NotFound desc = could not find container \"f70ccdbcb4556c24c3d22797c2a26cab5abb46a7ebfd682623f629f099c7c26f\": container with ID starting with f70ccdbcb4556c24c3d22797c2a26cab5abb46a7ebfd682623f629f099c7c26f not found: ID does not exist"
Sep 16 19:06:34.133525 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.133490    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"]
Sep 16 19:06:34.147920 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.147877    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-c190a0-openshift-default-cdbcc5fb-hg8hg"]
Sep 16 19:06:34.710634 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.710598    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c1295f7-3e69-4025-9381-fa348f881123" path="/var/lib/kubelet/pods/5c1295f7-3e69-4025-9381-fa348f881123/volumes"
Sep 16 19:06:34.800611 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.800577    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 16 19:06:34.800787 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:34.800648    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r" podUID="a223baca-28f9-49d2-bd57-798b4350d102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 16 19:06:35.801091 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:35.801052    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 16 19:06:35.801526 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:35.801132    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r" podUID="a223baca-28f9-49d2-bd57-798b4350d102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 16 19:06:36.806153 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:36.806120    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:06:36.806802 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:36.806775    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:06:36.807453 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:06:36.807432    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:07:07.335697 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:07.335600    2578 generic.go:356] "Generic (PLEG): container finished" podID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" containerID="86943eaf572e680baa52987808a6ada3527fdfef92875660333c80e6507366c8" exitCode=0
Sep 16 19:07:07.335697 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:07.335674    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql" event={"ID":"73c1758a-a532-4c93-9e2f-c2236f5d58b3","Type":"ContainerDied","Data":"86943eaf572e680baa52987808a6ada3527fdfef92875660333c80e6507366c8"}
Sep 16 19:07:07.336164 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:07.336147    2578 scope.go:117] "RemoveContainer" containerID="86943eaf572e680baa52987808a6ada3527fdfef92875660333c80e6507366c8"
Sep 16 19:07:07.337466 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:07.337433    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:07:07.337530 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:07.337510    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:07:07.337567 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:07.337544    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:07:08.349012 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:08.348975    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql" event={"ID":"73c1758a-a532-4c93-9e2f-c2236f5d58b3","Type":"ContainerStarted","Data":"e4d91c4bc51dbb4a0027740e624f6d8a205ec31b987855145ffbd959697fe906"}
Sep 16 19:07:08.734001 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:08.733966    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:07:08.735088 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:08.735063    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 16 19:07:08.735149 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:08.735115    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql" podUID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 16 19:07:09.733592 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:09.733556    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 16 19:07:09.733970 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:09.733619    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql" podUID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 16 19:07:10.733896 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:10.733863    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 16 19:07:10.734302 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:10.733912    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql" podUID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 16 19:07:11.199768 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.199733    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"]
Sep 16 19:07:11.200238 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.200222    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c1295f7-3e69-4025-9381-fa348f881123" containerName="istio-proxy"
Sep 16 19:07:11.200300 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.200239    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c1295f7-3e69-4025-9381-fa348f881123" containerName="istio-proxy"
Sep 16 19:07:11.200338 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.200329    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c1295f7-3e69-4025-9381-fa348f881123" containerName="istio-proxy"
Sep 16 19:07:11.202533 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.202517    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.207347 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.207310    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-d929c607\"/\"e2e-lifecycle-model-d929c607-kserve-self-signed-certs\""
Sep 16 19:07:11.207347 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.207324    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-d929c607\"/\"openshift-service-ca.crt\""
Sep 16 19:07:11.207544 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.207324    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-d929c607\"/\"default-dockercfg-997s4\""
Sep 16 19:07:11.207544 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.207322    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-d929c607\"/\"kube-root-ca.crt\""
Sep 16 19:07:11.222925 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.222898    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"]
Sep 16 19:07:11.313041 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.313000    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-home\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.313255 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.313050    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-dshm\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.313255 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.313111    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-tls-certs\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.313255 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.313159    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-model-cache\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.313255 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.313249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8mqj\" (UniqueName: \"kubernetes.io/projected/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-kube-api-access-c8mqj\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.313413 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.313277    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-tmp-dir\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.414665 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.414632    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-home\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.414836 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.414682    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-dshm\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.414836 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.414720    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-tls-certs\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.414836 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.414750    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-model-cache\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.414836 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.414798    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8mqj\" (UniqueName: \"kubernetes.io/projected/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-kube-api-access-c8mqj\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.414836 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.414826    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-tmp-dir\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.415146 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.415077    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-home\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.415254 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.415207    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-model-cache\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.415254 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.415239    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-tmp-dir\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.417047 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.417026    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-dshm\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.417362 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.417338    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-tls-certs\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.424032 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.424012    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8mqj\" (UniqueName: \"kubernetes.io/projected/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-kube-api-access-c8mqj\") pod \"e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") " pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.511649 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.511558    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:11.734306 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.734269    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 16 19:07:11.734681 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.734326    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql" podUID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 16 19:07:11.814035 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:11.814003    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"]
Sep 16 19:07:11.815513 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:07:11.815482    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4d3dc83_f9bc_44d4_a6d4_d7b3776601c4.slice/crio-3560a6d044acc5c952613cbbe09e9a9d0b5942c22f1e2fe205a951a06fd68486 WatchSource:0}: Error finding container 3560a6d044acc5c952613cbbe09e9a9d0b5942c22f1e2fe205a951a06fd68486: Status 404 returned error can't find the container with id 3560a6d044acc5c952613cbbe09e9a9d0b5942c22f1e2fe205a951a06fd68486
Sep 16 19:07:12.389140 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:12.389098    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r" event={"ID":"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4","Type":"ContainerStarted","Data":"223fe7b24b853535c570d4266dba82306f043073ea2b6dc20c2b3ba3b8bae3b8"}
Sep 16 19:07:12.389140 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:12.389148    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r" event={"ID":"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4","Type":"ContainerStarted","Data":"3560a6d044acc5c952613cbbe09e9a9d0b5942c22f1e2fe205a951a06fd68486"}
Sep 16 19:07:12.408640 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:12.408573    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r" podStartSLOduration=1.4085543600000001 podStartE2EDuration="1.40855436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:07:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:07:12.408320586 +0000 UTC m=+1660.195884762" watchObservedRunningTime="2026-09-16 19:07:12.40855436 +0000 UTC m=+1660.196118519"
Sep 16 19:07:12.734295 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:12.734205    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 16 19:07:12.734295 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:12.734260    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql" podUID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 16 19:07:13.036249 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:13.036133    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:07:13.734530 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:13.734490    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Sep 16 19:07:13.734911 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:13.734553    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql" podUID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Sep 16 19:07:14.738338 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:14.738309    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:07:14.739496 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:14.739472    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:07:15.884550 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:15.884516    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"]
Sep 16 19:07:20.177742 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.177696    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"]
Sep 16 19:07:20.180531 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.180508    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.183047 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.183021    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ca3f9a\"/\"openshift-service-ca.crt\""
Sep 16 19:07:20.183201 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.183061    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ca3f9a\"/\"kube-root-ca.crt\""
Sep 16 19:07:20.183688 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.183668    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ca3f9a\"/\"auth-test-model-ca3f9a-kserve-self-signed-certs\""
Sep 16 19:07:20.183741 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.183684    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ca3f9a\"/\"default-dockercfg-wfs2j\""
Sep 16 19:07:20.195910 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.195884    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"]
Sep 16 19:07:20.312962 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.312904    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-model-cache\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.312962 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.312966    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-home\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.313221 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.312994    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2e983d4-f3c1-495e-af59-d149ef1b57c1-tls-certs\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.313221 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.313050    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jpm8\" (UniqueName: \"kubernetes.io/projected/c2e983d4-f3c1-495e-af59-d149ef1b57c1-kube-api-access-7jpm8\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.313221 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.313099    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-tmp-dir\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.313387 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.313260    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-dshm\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.414606 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.414566    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-dshm\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.414785 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.414625    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-model-cache\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.414785 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.414648    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-home\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.414785 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.414669    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2e983d4-f3c1-495e-af59-d149ef1b57c1-tls-certs\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.414785 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.414714    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jpm8\" (UniqueName: \"kubernetes.io/projected/c2e983d4-f3c1-495e-af59-d149ef1b57c1-kube-api-access-7jpm8\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.414785 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.414759    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-tmp-dir\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.415077 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.415046    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-model-cache\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.415077 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.415066    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-home\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.415284 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.415246    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-tmp-dir\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.417036 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.417010    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-dshm\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.417360 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.417334    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2e983d4-f3c1-495e-af59-d149ef1b57c1-tls-certs\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.427640 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.427614    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jpm8\" (UniqueName: \"kubernetes.io/projected/c2e983d4-f3c1-495e-af59-d149ef1b57c1-kube-api-access-7jpm8\") pod \"auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") " pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.490988 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.490894    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:20.794152 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:20.794126    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"]
Sep 16 19:07:20.794961 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:07:20.794937    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2e983d4_f3c1_495e_af59_d149ef1b57c1.slice/crio-ceb3055061c9f74aec6762cc491694d15229441a08642885ca7da1246caf4eb8 WatchSource:0}: Error finding container ceb3055061c9f74aec6762cc491694d15229441a08642885ca7da1246caf4eb8: Status 404 returned error can't find the container with id ceb3055061c9f74aec6762cc491694d15229441a08642885ca7da1246caf4eb8
Sep 16 19:07:21.454758 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:21.454722    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm" event={"ID":"c2e983d4-f3c1-495e-af59-d149ef1b57c1","Type":"ContainerStarted","Data":"a5b89e5e8e48e01b82d1758356b4e4a905d2bcf007d8bb3213beb84b3a0e48c4"}
Sep 16 19:07:21.454758 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:21.454765    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm" event={"ID":"c2e983d4-f3c1-495e-af59-d149ef1b57c1","Type":"ContainerStarted","Data":"ceb3055061c9f74aec6762cc491694d15229441a08642885ca7da1246caf4eb8"}
Sep 16 19:07:21.481207 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:21.481134    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm" podStartSLOduration=1.481117038 podStartE2EDuration="1.481117038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:07:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:07:21.478271066 +0000 UTC m=+1669.265835224" watchObservedRunningTime="2026-09-16 19:07:21.481117038 +0000 UTC m=+1669.268681210"
Sep 16 19:07:21.512374 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:21.512334    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:21.513624 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:21.513573    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:21.552701 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:21.552671    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:22.473462 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:22.473433    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:07:30.491523 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:30.491481    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:30.491523 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:30.491533    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:30.504500 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:30.504474    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:30.535136 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:30.535108    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:07:30.904108 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:30.904042    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z" podUID="2eefc8c5-0b5a-4666-a4ec-1288f0712a70" containerName="main" containerID="cri-o://9820f8395f7149a6ad9bece330c03436a557ea73e5bb38a86246ab0afe54d77e" gracePeriod=45
Sep 16 19:07:31.160196 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.160112    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:07:31.339703 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.339665    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-home\") pod \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") "
Sep 16 19:07:31.339904 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.339747    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-dshm\") pod \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") "
Sep 16 19:07:31.339904 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.339764    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-model-cache\") pod \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") "
Sep 16 19:07:31.339904 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.339786    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-tmp-dir\") pod \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") "
Sep 16 19:07:31.339904 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.339820    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wkqn\" (UniqueName: \"kubernetes.io/projected/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-kube-api-access-8wkqn\") pod \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") "
Sep 16 19:07:31.339904 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.339840    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-tls-certs\") pod \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\" (UID: \"2eefc8c5-0b5a-4666-a4ec-1288f0712a70\") "
Sep 16 19:07:31.340232 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.340016    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-home" (OuterVolumeSpecName: "home") pod "2eefc8c5-0b5a-4666-a4ec-1288f0712a70" (UID: "2eefc8c5-0b5a-4666-a4ec-1288f0712a70"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:07:31.340232 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.340036    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-model-cache" (OuterVolumeSpecName: "model-cache") pod "2eefc8c5-0b5a-4666-a4ec-1288f0712a70" (UID: "2eefc8c5-0b5a-4666-a4ec-1288f0712a70"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:07:31.340401 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.340250    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-model-cache\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:07:31.340401 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.340267    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-home\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:07:31.340401 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.340356    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2eefc8c5-0b5a-4666-a4ec-1288f0712a70" (UID: "2eefc8c5-0b5a-4666-a4ec-1288f0712a70"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:07:31.342267 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.342243    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2eefc8c5-0b5a-4666-a4ec-1288f0712a70" (UID: "2eefc8c5-0b5a-4666-a4ec-1288f0712a70"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:07:31.342520 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.342500    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-dshm" (OuterVolumeSpecName: "dshm") pod "2eefc8c5-0b5a-4666-a4ec-1288f0712a70" (UID: "2eefc8c5-0b5a-4666-a4ec-1288f0712a70"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:07:31.342602 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.342534    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-kube-api-access-8wkqn" (OuterVolumeSpecName: "kube-api-access-8wkqn") pod "2eefc8c5-0b5a-4666-a4ec-1288f0712a70" (UID: "2eefc8c5-0b5a-4666-a4ec-1288f0712a70"). InnerVolumeSpecName "kube-api-access-8wkqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:07:31.441092 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.440986    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-dshm\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:07:31.441092 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.441024    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-tmp-dir\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:07:31.441092 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.441039    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wkqn\" (UniqueName: \"kubernetes.io/projected/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-kube-api-access-8wkqn\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:07:31.441092 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.441050    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2eefc8c5-0b5a-4666-a4ec-1288f0712a70-tls-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:07:31.530226 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.530197    2578 generic.go:356] "Generic (PLEG): container finished" podID="2eefc8c5-0b5a-4666-a4ec-1288f0712a70" containerID="9820f8395f7149a6ad9bece330c03436a557ea73e5bb38a86246ab0afe54d77e" exitCode=0
Sep 16 19:07:31.530651 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.530281    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"
Sep 16 19:07:31.530651 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.530321    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z" event={"ID":"2eefc8c5-0b5a-4666-a4ec-1288f0712a70","Type":"ContainerDied","Data":"9820f8395f7149a6ad9bece330c03436a557ea73e5bb38a86246ab0afe54d77e"}
Sep 16 19:07:31.530651 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.530369    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z" event={"ID":"2eefc8c5-0b5a-4666-a4ec-1288f0712a70","Type":"ContainerDied","Data":"4d099c98fb04d0bbc71f37e48c15ba2724019c288987f4018ecd95b551a5ddfd"}
Sep 16 19:07:31.530651 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.530392    2578 scope.go:117] "RemoveContainer" containerID="9820f8395f7149a6ad9bece330c03436a557ea73e5bb38a86246ab0afe54d77e"
Sep 16 19:07:31.541826 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.541803    2578 scope.go:117] "RemoveContainer" containerID="9820f8395f7149a6ad9bece330c03436a557ea73e5bb38a86246ab0afe54d77e"
Sep 16 19:07:31.542198 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:07:31.542157    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9820f8395f7149a6ad9bece330c03436a557ea73e5bb38a86246ab0afe54d77e\": container with ID starting with 9820f8395f7149a6ad9bece330c03436a557ea73e5bb38a86246ab0afe54d77e not found: ID does not exist" containerID="9820f8395f7149a6ad9bece330c03436a557ea73e5bb38a86246ab0afe54d77e"
Sep 16 19:07:31.542275 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.542209    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9820f8395f7149a6ad9bece330c03436a557ea73e5bb38a86246ab0afe54d77e"} err="failed to get container status \"9820f8395f7149a6ad9bece330c03436a557ea73e5bb38a86246ab0afe54d77e\": rpc error: code = NotFound desc = could not find container \"9820f8395f7149a6ad9bece330c03436a557ea73e5bb38a86246ab0afe54d77e\": container with ID starting with 9820f8395f7149a6ad9bece330c03436a557ea73e5bb38a86246ab0afe54d77e not found: ID does not exist"
Sep 16 19:07:31.553243 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.553212    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"]
Sep 16 19:07:31.559130 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:31.559108    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-c090c8/e2e-conflict-b-05590707-kserve-584c947b5b-mqs5z"]
Sep 16 19:07:32.710283 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:32.710252    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2eefc8c5-0b5a-4666-a4ec-1288f0712a70" path="/var/lib/kubelet/pods/2eefc8c5-0b5a-4666-a4ec-1288f0712a70/volumes"
Sep 16 19:07:33.020447 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:33.020366    2578 scope.go:117] "RemoveContainer" containerID="ea31bd8a3d7709b29b2fc5701accc5e8d916dad173037fc72a310b537108f20b"
Sep 16 19:07:52.111956 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:52.111917    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d929c607-gl5nq"]
Sep 16 19:07:52.112488 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:52.112412    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eefc8c5-0b5a-4666-a4ec-1288f0712a70" containerName="main"
Sep 16 19:07:52.112488 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:52.112425    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eefc8c5-0b5a-4666-a4ec-1288f0712a70" containerName="main"
Sep 16 19:07:52.112488 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:52.112487    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2eefc8c5-0b5a-4666-a4ec-1288f0712a70" containerName="main"
Sep 16 19:07:52.114707 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:52.114690    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d929c607-gl5nq"
Sep 16 19:07:52.117848 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:52.117823    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nmr62\""
Sep 16 19:07:52.125814 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:52.125791    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d929c607-gl5nq"]
Sep 16 19:07:52.181799 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:52.181767    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b37bd76-4484-4580-ac87-671c6f937256-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d929c607-gl5nq\" (UID: \"2b37bd76-4484-4580-ac87-671c6f937256\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d929c607-gl5nq"
Sep 16 19:07:52.283250 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:52.283209    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b37bd76-4484-4580-ac87-671c6f937256-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d929c607-gl5nq\" (UID: \"2b37bd76-4484-4580-ac87-671c6f937256\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d929c607-gl5nq"
Sep 16 19:07:52.283862 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:52.283834    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b37bd76-4484-4580-ac87-671c6f937256-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-d929c607-gl5nq\" (UID: \"2b37bd76-4484-4580-ac87-671c6f937256\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d929c607-gl5nq"
Sep 16 19:07:52.427085 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:52.426985    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d929c607-gl5nq"
Sep 16 19:07:52.709604 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:07:52.709571    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b37bd76_4484_4580_ac87_671c6f937256.slice/crio-0eccf50d73a67b0bd25ad14b4d4e2c5fd46cac4125ee006e2e89d30f7dea9ec3 WatchSource:0}: Error finding container 0eccf50d73a67b0bd25ad14b4d4e2c5fd46cac4125ee006e2e89d30f7dea9ec3: Status 404 returned error can't find the container with id 0eccf50d73a67b0bd25ad14b4d4e2c5fd46cac4125ee006e2e89d30f7dea9ec3
Sep 16 19:07:52.711310 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:52.711291    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d929c607-gl5nq"]
Sep 16 19:07:53.683321 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:53.683288    2578 generic.go:356] "Generic (PLEG): container finished" podID="2b37bd76-4484-4580-ac87-671c6f937256" containerID="33b433eb437803b68971773a4bcd1c80b9bad976b61a276bc31a767a53b1ab0e" exitCode=0
Sep 16 19:07:53.683779 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:53.683351    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d929c607-gl5nq" event={"ID":"2b37bd76-4484-4580-ac87-671c6f937256","Type":"ContainerDied","Data":"33b433eb437803b68971773a4bcd1c80b9bad976b61a276bc31a767a53b1ab0e"}
Sep 16 19:07:53.683779 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:53.683389    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d929c607-gl5nq" event={"ID":"2b37bd76-4484-4580-ac87-671c6f937256","Type":"ContainerStarted","Data":"0eccf50d73a67b0bd25ad14b4d4e2c5fd46cac4125ee006e2e89d30f7dea9ec3"}
Sep 16 19:07:54.837004 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:54.836973    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d929c607-gl5nq"
Sep 16 19:07:54.919825 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:54.919743    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b37bd76-4484-4580-ac87-671c6f937256-maas-api-service-ca\") pod \"2b37bd76-4484-4580-ac87-671c6f937256\" (UID: \"2b37bd76-4484-4580-ac87-671c6f937256\") "
Sep 16 19:07:54.920165 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:54.920140    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b37bd76-4484-4580-ac87-671c6f937256-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2b37bd76-4484-4580-ac87-671c6f937256" (UID: "2b37bd76-4484-4580-ac87-671c6f937256"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:07:54.920284 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:54.920269    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b37bd76-4484-4580-ac87-671c6f937256-maas-api-service-ca\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:07:55.697979 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:55.697949    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d929c607-gl5nq"
Sep 16 19:07:55.698158 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:55.697946    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d929c607-gl5nq" event={"ID":"2b37bd76-4484-4580-ac87-671c6f937256","Type":"ContainerDied","Data":"0eccf50d73a67b0bd25ad14b4d4e2c5fd46cac4125ee006e2e89d30f7dea9ec3"}
Sep 16 19:07:55.698158 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:07:55.698081    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0eccf50d73a67b0bd25ad14b4d4e2c5fd46cac4125ee006e2e89d30f7dea9ec3"
Sep 16 19:08:03.479474 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.479441    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"]
Sep 16 19:08:03.480185 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.480150    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b37bd76-4484-4580-ac87-671c6f937256" containerName="revoke-keys"
Sep 16 19:08:03.480301 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.480187    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b37bd76-4484-4580-ac87-671c6f937256" containerName="revoke-keys"
Sep 16 19:08:03.480361 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.480299    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b37bd76-4484-4580-ac87-671c6f937256" containerName="revoke-keys"
Sep 16 19:08:03.483675 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.483653    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.488263 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.488242    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-2dc9fc\"/\"default-dockercfg-g7tk4\""
Sep 16 19:08:03.488398 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.488273    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-2dc9fc\"/\"kube-root-ca.crt\""
Sep 16 19:08:03.488398 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.488292    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-2dc9fc\"/\"auth-test-model-2dc9fc-kserve-self-signed-certs\""
Sep 16 19:08:03.488398 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.488243    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-2dc9fc\"/\"openshift-service-ca.crt\""
Sep 16 19:08:03.505686 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.505654    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"]
Sep 16 19:08:03.515648 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.515618    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvx4t\" (UniqueName: \"kubernetes.io/projected/44a6241f-65d7-465d-9300-5237a7c871b3-kube-api-access-qvx4t\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.515827 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.515664    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44a6241f-65d7-465d-9300-5237a7c871b3-tls-certs\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.515827 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.515745    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-tmp-dir\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.515827 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.515781    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-model-cache\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.515827 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.515799    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-home\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.516005 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.515882    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-dshm\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.617394 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.617357    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44a6241f-65d7-465d-9300-5237a7c871b3-tls-certs\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.617564 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.617422    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-tmp-dir\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.617564 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.617448    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-model-cache\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.617564 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.617471    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-home\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.617564 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.617526    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-dshm\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.617791 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.617671    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvx4t\" (UniqueName: \"kubernetes.io/projected/44a6241f-65d7-465d-9300-5237a7c871b3-kube-api-access-qvx4t\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.617934 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.617909    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-model-cache\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.618009 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.617950    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-tmp-dir\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.618009 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.617967    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-home\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.619831 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.619802    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-dshm\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.619959 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.619875    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44a6241f-65d7-465d-9300-5237a7c871b3-tls-certs\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.630754 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.630731    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvx4t\" (UniqueName: \"kubernetes.io/projected/44a6241f-65d7-465d-9300-5237a7c871b3-kube-api-access-qvx4t\") pod \"auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") " pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:03.793744 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:03.793653    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:04.077252 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:04.077227    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"]
Sep 16 19:08:04.078570 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:08:04.078531    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44a6241f_65d7_465d_9300_5237a7c871b3.slice/crio-075c93f3ddf3996eb0f3032288dda7c92e7373f6f9756aeb6a9e6d10f23f457b WatchSource:0}: Error finding container 075c93f3ddf3996eb0f3032288dda7c92e7373f6f9756aeb6a9e6d10f23f457b: Status 404 returned error can't find the container with id 075c93f3ddf3996eb0f3032288dda7c92e7373f6f9756aeb6a9e6d10f23f457b
Sep 16 19:08:04.765459 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:04.765416    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v" event={"ID":"44a6241f-65d7-465d-9300-5237a7c871b3","Type":"ContainerStarted","Data":"f53122563a9b5c5e0229869ac2cebb65901a88cb78e6b76c0c5eca8287d2c8a3"}
Sep 16 19:08:04.765977 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:04.765471    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v" event={"ID":"44a6241f-65d7-465d-9300-5237a7c871b3","Type":"ContainerStarted","Data":"075c93f3ddf3996eb0f3032288dda7c92e7373f6f9756aeb6a9e6d10f23f457b"}
Sep 16 19:08:04.792555 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:04.792498    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v" podStartSLOduration=1.79247779 podStartE2EDuration="1.79247779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:08:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:08:04.789475703 +0000 UTC m=+1712.577039858" watchObservedRunningTime="2026-09-16 19:08:04.79247779 +0000 UTC m=+1712.580041945"
Sep 16 19:08:08.062944 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:08.062906    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d929c607-gl5nq"]
Sep 16 19:08:08.067850 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:08.067823    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-d929c607-gl5nq"]
Sep 16 19:08:08.710003 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:08.709970    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b37bd76-4484-4580-ac87-671c6f937256" path="/var/lib/kubelet/pods/2b37bd76-4484-4580-ac87-671c6f937256/volumes"
Sep 16 19:08:13.794061 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:13.794025    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:13.794595 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:13.794078    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:13.806527 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:13.806503    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:13.841559 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:13.841525    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:08:16.451361 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:16.451321    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"]
Sep 16 19:08:31.471019 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.470975    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r" podUID="e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4" containerName="main" containerID="cri-o://223fe7b24b853535c570d4266dba82306f043073ea2b6dc20c2b3ba3b8bae3b8" gracePeriod=45
Sep 16 19:08:31.708871 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.708845    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:08:31.739662 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.739568    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-dshm\") pod \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") "
Sep 16 19:08:31.739662 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.739621    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8mqj\" (UniqueName: \"kubernetes.io/projected/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-kube-api-access-c8mqj\") pod \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") "
Sep 16 19:08:31.740138 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.740111    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-tmp-dir\") pod \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") "
Sep 16 19:08:31.740263 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.740163    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-home\") pod \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") "
Sep 16 19:08:31.740263 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.740231    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-model-cache\") pod \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") "
Sep 16 19:08:31.740367 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.740294    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-tls-certs\") pod \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\" (UID: \"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4\") "
Sep 16 19:08:31.740799 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.740680    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4" (UID: "e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:08:31.740799 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.740752    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-home" (OuterVolumeSpecName: "home") pod "e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4" (UID: "e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:08:31.743014 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.741280    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-model-cache" (OuterVolumeSpecName: "model-cache") pod "e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4" (UID: "e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:08:31.743157 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.743136    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-dshm" (OuterVolumeSpecName: "dshm") pod "e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4" (UID: "e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:08:31.744189 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.744151    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4" (UID: "e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:08:31.744286 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.744259    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-kube-api-access-c8mqj" (OuterVolumeSpecName: "kube-api-access-c8mqj") pod "e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4" (UID: "e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4"). InnerVolumeSpecName "kube-api-access-c8mqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:08:31.841366 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.841322    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-tls-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:08:31.841366 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.841364    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-dshm\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:08:31.841366 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.841375    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8mqj\" (UniqueName: \"kubernetes.io/projected/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-kube-api-access-c8mqj\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:08:31.841609 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.841385    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-tmp-dir\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:08:31.841609 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.841395    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-home\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:08:31.841609 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.841403    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4-model-cache\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:08:31.956968 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.956928    2578 generic.go:356] "Generic (PLEG): container finished" podID="e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4" containerID="223fe7b24b853535c570d4266dba82306f043073ea2b6dc20c2b3ba3b8bae3b8" exitCode=0
Sep 16 19:08:31.957137 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.956977    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"
Sep 16 19:08:31.957137 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.957023    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r" event={"ID":"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4","Type":"ContainerDied","Data":"223fe7b24b853535c570d4266dba82306f043073ea2b6dc20c2b3ba3b8bae3b8"}
Sep 16 19:08:31.957137 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.957066    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r" event={"ID":"e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4","Type":"ContainerDied","Data":"3560a6d044acc5c952613cbbe09e9a9d0b5942c22f1e2fe205a951a06fd68486"}
Sep 16 19:08:31.957137 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.957087    2578 scope.go:117] "RemoveContainer" containerID="223fe7b24b853535c570d4266dba82306f043073ea2b6dc20c2b3ba3b8bae3b8"
Sep 16 19:08:31.967371 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.967324    2578 scope.go:117] "RemoveContainer" containerID="223fe7b24b853535c570d4266dba82306f043073ea2b6dc20c2b3ba3b8bae3b8"
Sep 16 19:08:31.967647 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:08:31.967628    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"223fe7b24b853535c570d4266dba82306f043073ea2b6dc20c2b3ba3b8bae3b8\": container with ID starting with 223fe7b24b853535c570d4266dba82306f043073ea2b6dc20c2b3ba3b8bae3b8 not found: ID does not exist" containerID="223fe7b24b853535c570d4266dba82306f043073ea2b6dc20c2b3ba3b8bae3b8"
Sep 16 19:08:31.967693 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.967655    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"223fe7b24b853535c570d4266dba82306f043073ea2b6dc20c2b3ba3b8bae3b8"} err="failed to get container status \"223fe7b24b853535c570d4266dba82306f043073ea2b6dc20c2b3ba3b8bae3b8\": rpc error: code = NotFound desc = could not find container \"223fe7b24b853535c570d4266dba82306f043073ea2b6dc20c2b3ba3b8bae3b8\": container with ID starting with 223fe7b24b853535c570d4266dba82306f043073ea2b6dc20c2b3ba3b8bae3b8 not found: ID does not exist"
Sep 16 19:08:31.992061 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.991996    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"]
Sep 16 19:08:31.995985 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:31.995951    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-d929c607/e2e-lifecycle-model-d929c607-kserve-57f9b4b97f-xzv5r"]
Sep 16 19:08:32.712056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:08:32.712019    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4" path="/var/lib/kubelet/pods/e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4/volumes"
Sep 16 19:09:02.864102 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:02.864067    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"]
Sep 16 19:09:02.864686 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:02.864332    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4" podUID="2eb14464-0bdf-46f4-9ac3-9193779cfad5" containerName="istio-proxy" containerID="cri-o://1155217ee189e821cb1e5965da9f1b9be831ada7af4bc467578caecb5392ad41" gracePeriod=30
Sep 16 19:09:08.124682 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.124652    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:09:08.207834 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.207750    2578 generic.go:356] "Generic (PLEG): container finished" podID="2eb14464-0bdf-46f4-9ac3-9193779cfad5" containerID="1155217ee189e821cb1e5965da9f1b9be831ada7af4bc467578caecb5392ad41" exitCode=0
Sep 16 19:09:08.207834 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.207797    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"
Sep 16 19:09:08.207834 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.207820    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4" event={"ID":"2eb14464-0bdf-46f4-9ac3-9193779cfad5","Type":"ContainerDied","Data":"1155217ee189e821cb1e5965da9f1b9be831ada7af4bc467578caecb5392ad41"}
Sep 16 19:09:08.208050 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.207869    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4" event={"ID":"2eb14464-0bdf-46f4-9ac3-9193779cfad5","Type":"ContainerDied","Data":"b1611d99276ee9f84d91a4a20c61b2b5876ab8500fe7bb8dd896ec612ebc58af"}
Sep 16 19:09:08.208050 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.207894    2578 scope.go:117] "RemoveContainer" containerID="1155217ee189e821cb1e5965da9f1b9be831ada7af4bc467578caecb5392ad41"
Sep 16 19:09:08.217954 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.217929    2578 scope.go:117] "RemoveContainer" containerID="b31afa33ddea5651910a58bbaa753efefbc4915c191d0b5075169990d05e5c86"
Sep 16 19:09:08.227729 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.227709    2578 scope.go:117] "RemoveContainer" containerID="1155217ee189e821cb1e5965da9f1b9be831ada7af4bc467578caecb5392ad41"
Sep 16 19:09:08.228030 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:09:08.228011    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1155217ee189e821cb1e5965da9f1b9be831ada7af4bc467578caecb5392ad41\": container with ID starting with 1155217ee189e821cb1e5965da9f1b9be831ada7af4bc467578caecb5392ad41 not found: ID does not exist" containerID="1155217ee189e821cb1e5965da9f1b9be831ada7af4bc467578caecb5392ad41"
Sep 16 19:09:08.228079 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.228039    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1155217ee189e821cb1e5965da9f1b9be831ada7af4bc467578caecb5392ad41"} err="failed to get container status \"1155217ee189e821cb1e5965da9f1b9be831ada7af4bc467578caecb5392ad41\": rpc error: code = NotFound desc = could not find container \"1155217ee189e821cb1e5965da9f1b9be831ada7af4bc467578caecb5392ad41\": container with ID starting with 1155217ee189e821cb1e5965da9f1b9be831ada7af4bc467578caecb5392ad41 not found: ID does not exist"
Sep 16 19:09:08.228079 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.228057    2578 scope.go:117] "RemoveContainer" containerID="b31afa33ddea5651910a58bbaa753efefbc4915c191d0b5075169990d05e5c86"
Sep 16 19:09:08.228355 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:09:08.228336    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b31afa33ddea5651910a58bbaa753efefbc4915c191d0b5075169990d05e5c86\": container with ID starting with b31afa33ddea5651910a58bbaa753efefbc4915c191d0b5075169990d05e5c86 not found: ID does not exist" containerID="b31afa33ddea5651910a58bbaa753efefbc4915c191d0b5075169990d05e5c86"
Sep 16 19:09:08.228427 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.228359    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b31afa33ddea5651910a58bbaa753efefbc4915c191d0b5075169990d05e5c86"} err="failed to get container status \"b31afa33ddea5651910a58bbaa753efefbc4915c191d0b5075169990d05e5c86\": rpc error: code = NotFound desc = could not find container \"b31afa33ddea5651910a58bbaa753efefbc4915c191d0b5075169990d05e5c86\": container with ID starting with b31afa33ddea5651910a58bbaa753efefbc4915c191d0b5075169990d05e5c86 not found: ID does not exist"
Sep 16 19:09:08.239776 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.239751    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-credential-socket\") pod \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") "
Sep 16 19:09:08.239920 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.239839    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-podinfo\") pod \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") "
Sep 16 19:09:08.239920 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.239914    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-token\") pod \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") "
Sep 16 19:09:08.240035 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.240007    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4x95\" (UniqueName: \"kubernetes.io/projected/2eb14464-0bdf-46f4-9ac3-9193779cfad5-kube-api-access-d4x95\") pod \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") "
Sep 16 19:09:08.240090 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.240049    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istiod-ca-cert\") pod \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") "
Sep 16 19:09:08.240090 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.240059    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2eb14464-0bdf-46f4-9ac3-9193779cfad5" (UID: "2eb14464-0bdf-46f4-9ac3-9193779cfad5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:09:08.240233 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.240087    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-workload-certs\") pod \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") "
Sep 16 19:09:08.240233 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.240161    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-workload-socket\") pod \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") "
Sep 16 19:09:08.240233 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.240208    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-envoy\") pod \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") "
Sep 16 19:09:08.240890 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.240238    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-data\") pod \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\" (UID: \"2eb14464-0bdf-46f4-9ac3-9193779cfad5\") "
Sep 16 19:09:08.240890 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.240409    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2eb14464-0bdf-46f4-9ac3-9193779cfad5" (UID: "2eb14464-0bdf-46f4-9ac3-9193779cfad5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:09:08.240890 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.240491    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2eb14464-0bdf-46f4-9ac3-9193779cfad5" (UID: "2eb14464-0bdf-46f4-9ac3-9193779cfad5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:09:08.240890 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.240673    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-data" (OuterVolumeSpecName: "istio-data") pod "2eb14464-0bdf-46f4-9ac3-9193779cfad5" (UID: "2eb14464-0bdf-46f4-9ac3-9193779cfad5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:09:08.240890 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.240795    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:09:08.240890 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.240811    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:09:08.240890 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.240823    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:09:08.240890 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.240835    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:09:08.241420 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.241248    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2eb14464-0bdf-46f4-9ac3-9193779cfad5" (UID: "2eb14464-0bdf-46f4-9ac3-9193779cfad5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:09:08.242761 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.242723    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2eb14464-0bdf-46f4-9ac3-9193779cfad5" (UID: "2eb14464-0bdf-46f4-9ac3-9193779cfad5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:09:08.242761 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.242751    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-token" (OuterVolumeSpecName: "istio-token") pod "2eb14464-0bdf-46f4-9ac3-9193779cfad5" (UID: "2eb14464-0bdf-46f4-9ac3-9193779cfad5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:09:08.243055 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.243026    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eb14464-0bdf-46f4-9ac3-9193779cfad5-kube-api-access-d4x95" (OuterVolumeSpecName: "kube-api-access-d4x95") pod "2eb14464-0bdf-46f4-9ac3-9193779cfad5" (UID: "2eb14464-0bdf-46f4-9ac3-9193779cfad5"). InnerVolumeSpecName "kube-api-access-d4x95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:09:08.243629 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.243599    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2eb14464-0bdf-46f4-9ac3-9193779cfad5" (UID: "2eb14464-0bdf-46f4-9ac3-9193779cfad5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:09:08.342264 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.342222    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:09:08.342264 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.342259    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:09:08.342467 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.342274    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4x95\" (UniqueName: \"kubernetes.io/projected/2eb14464-0bdf-46f4-9ac3-9193779cfad5-kube-api-access-d4x95\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:09:08.342467 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.342290    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:09:08.342467 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.342322    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2eb14464-0bdf-46f4-9ac3-9193779cfad5-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:09:08.538225 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.538186    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"]
Sep 16 19:09:08.542485 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.542456    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-d929c607-openshift-default-5d6b4f8996-shnw4"]
Sep 16 19:09:08.710721 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:08.710685    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2eb14464-0bdf-46f4-9ac3-9193779cfad5" path="/var/lib/kubelet/pods/2eb14464-0bdf-46f4-9ac3-9193779cfad5/volumes"
Sep 16 19:09:17.395508 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.395471    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"]
Sep 16 19:09:17.395976 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.395962    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eb14464-0bdf-46f4-9ac3-9193779cfad5" containerName="istio-proxy"
Sep 16 19:09:17.396031 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.395977    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eb14464-0bdf-46f4-9ac3-9193779cfad5" containerName="istio-proxy"
Sep 16 19:09:17.396031 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.395993    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eb14464-0bdf-46f4-9ac3-9193779cfad5" containerName="istio-proxy"
Sep 16 19:09:17.396031 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.395999    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eb14464-0bdf-46f4-9ac3-9193779cfad5" containerName="istio-proxy"
Sep 16 19:09:17.396031 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.396011    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4" containerName="main"
Sep 16 19:09:17.396031 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.396022    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4" containerName="main"
Sep 16 19:09:17.396203 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.396102    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4d3dc83-f9bc-44d4-a6d4-d7b3776601c4" containerName="main"
Sep 16 19:09:17.396203 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.396109    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2eb14464-0bdf-46f4-9ac3-9193779cfad5" containerName="istio-proxy"
Sep 16 19:09:17.396203 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.396114    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2eb14464-0bdf-46f4-9ac3-9193779cfad5" containerName="istio-proxy"
Sep 16 19:09:17.399282 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.399261    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.401996 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.401958    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-b1857c\"/\"embedding-w0-b1857c-kserve-self-signed-certs\""
Sep 16 19:09:17.402707 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.402685    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-b1857c\"/\"openshift-service-ca.crt\""
Sep 16 19:09:17.402818 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.402688    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-b1857c\"/\"kube-root-ca.crt\""
Sep 16 19:09:17.402818 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.402766    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-b1857c\"/\"default-dockercfg-9xd2j\""
Sep 16 19:09:17.420560 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.420533    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"]
Sep 16 19:09:17.542031 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.541991    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-dshm\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.542031 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.542031    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9855a54a-d7e8-42b1-aed2-a2ed874b152a-tls-certs\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.542315 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.542055    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-tmp-dir\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.542315 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.542155    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rtnl\" (UniqueName: \"kubernetes.io/projected/9855a54a-d7e8-42b1-aed2-a2ed874b152a-kube-api-access-9rtnl\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.542315 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.542223    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-model-cache\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.542460 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.542355    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-home\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.643443 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.643399    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-model-cache\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.643643 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.643533    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-home\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.643643 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.643562    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-dshm\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.643643 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.643579    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9855a54a-d7e8-42b1-aed2-a2ed874b152a-tls-certs\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.643643 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.643609    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-tmp-dir\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.643875 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.643655    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rtnl\" (UniqueName: \"kubernetes.io/projected/9855a54a-d7e8-42b1-aed2-a2ed874b152a-kube-api-access-9rtnl\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.643875 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.643856    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-model-cache\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.643987 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.643918    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-home\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.644134 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.644114    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-tmp-dir\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.646051 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.645982    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-dshm\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.646295 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.646276    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9855a54a-d7e8-42b1-aed2-a2ed874b152a-tls-certs\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.653109 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.653083    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rtnl\" (UniqueName: \"kubernetes.io/projected/9855a54a-d7e8-42b1-aed2-a2ed874b152a-kube-api-access-9rtnl\") pod \"embedding-w0-b1857c-kserve-569cbdf44d-5h2b2\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") " pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.709395 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.709364    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:17.987712 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:17.987680    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"]
Sep 16 19:09:17.988539 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:09:17.988516    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9855a54a_d7e8_42b1_aed2_a2ed874b152a.slice/crio-2a19ec597e5458c5a8a17c42bf930d8a682c515f05b79840a6ccd512429d0152 WatchSource:0}: Error finding container 2a19ec597e5458c5a8a17c42bf930d8a682c515f05b79840a6ccd512429d0152: Status 404 returned error can't find the container with id 2a19ec597e5458c5a8a17c42bf930d8a682c515f05b79840a6ccd512429d0152
Sep 16 19:09:18.274726 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:18.274633    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2" event={"ID":"9855a54a-d7e8-42b1-aed2-a2ed874b152a","Type":"ContainerStarted","Data":"0de98ec83b56be4ecb4a2d440b0046917bf2727e737258c33bb3c2ed1fe940eb"}
Sep 16 19:09:18.274726 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:18.274677    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2" event={"ID":"9855a54a-d7e8-42b1-aed2-a2ed874b152a","Type":"ContainerStarted","Data":"2a19ec597e5458c5a8a17c42bf930d8a682c515f05b79840a6ccd512429d0152"}
Sep 16 19:09:18.296059 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:18.296006    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2" podStartSLOduration=1.295988471 podStartE2EDuration="1.295988471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:09:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:09:18.29397159 +0000 UTC m=+1786.081535764" watchObservedRunningTime="2026-09-16 19:09:18.295988471 +0000 UTC m=+1786.083552658"
Sep 16 19:09:27.710393 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:27.710350    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:27.710772 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:27.710409    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:27.723483 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:27.723452    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:28.353402 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:28.353370    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:09:32.832834 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:32.832804    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 19:09:32.833969 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:32.833945    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 19:09:32.839425 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:32.839393    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 19:09:32.840462 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:09:32.840442    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 19:10:03.648617 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.648570    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"]
Sep 16 19:10:03.661016 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.660989    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.665919 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.665889    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a46230-openshift-default-dockercfg-bdwll\""
Sep 16 19:10:03.672660 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.672633    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"]
Sep 16 19:10:03.749889 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.749847    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-credential-socket\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.750245 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.750220    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wqcs\" (UniqueName: \"kubernetes.io/projected/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-kube-api-access-4wqcs\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.750480 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.750460    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-token\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.750654 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.750638    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-podinfo\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.750836 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.750821    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istiod-ca-cert\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.750990 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.750976    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-workload-certs\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.751163 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.751148    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-envoy\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.751350 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.751333    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-data\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.751518 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.751502    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-workload-socket\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.853048 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.853011    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-token\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.853253 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.853057    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-podinfo\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.853253 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.853091    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istiod-ca-cert\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.853253 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.853110    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-workload-certs\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.853440 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.853259    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-envoy\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.853440 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.853300    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-data\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.853440 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.853343    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-workload-socket\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.853440 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.853426    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-credential-socket\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.853653 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.853475    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wqcs\" (UniqueName: \"kubernetes.io/projected/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-kube-api-access-4wqcs\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.853797 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.853746    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-workload-certs\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.853929 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.853899    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-workload-socket\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.854071 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.853953    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-data\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.854071 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.853973    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istiod-ca-cert\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.854071 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.853984    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-credential-socket\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.855594 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.855574    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-envoy\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.855722 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.855705    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-podinfo\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.864749 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.864721    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-token\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.865559 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.865537    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wqcs\" (UniqueName: \"kubernetes.io/projected/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-kube-api-access-4wqcs\") pod \"e2e-api-a-a46230-openshift-default-56b789c668-qs2kz\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") " pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:03.977472 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:03.977380    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:04.281858 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:04.281830    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"]
Sep 16 19:10:04.283361 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:10:04.283325    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3a3aca2_0d4e_4b40_8b54_f4ca27b3b9a6.slice/crio-fedf3282ab8c71dbdc488614ddc57e2d27e9d0cd98b108ed82a4a4c7daa2c893 WatchSource:0}: Error finding container fedf3282ab8c71dbdc488614ddc57e2d27e9d0cd98b108ed82a4a4c7daa2c893: Status 404 returned error can't find the container with id fedf3282ab8c71dbdc488614ddc57e2d27e9d0cd98b108ed82a4a4c7daa2c893
Sep 16 19:10:04.285830 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:04.285794    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:10:04.285916 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:04.285867    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:10:04.285916 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:04.285903    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:10:04.590447 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:04.590369    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz" event={"ID":"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6","Type":"ContainerStarted","Data":"628f7237b8a9620e297e36f36efda7b8ed303441efde0c04db991b5881f50450"}
Sep 16 19:10:04.590447 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:04.590421    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz" event={"ID":"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6","Type":"ContainerStarted","Data":"fedf3282ab8c71dbdc488614ddc57e2d27e9d0cd98b108ed82a4a4c7daa2c893"}
Sep 16 19:10:04.628217 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:04.628131    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz" podStartSLOduration=1.628111912 podStartE2EDuration="1.628111912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:10:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:10:04.624465228 +0000 UTC m=+1832.412029405" watchObservedRunningTime="2026-09-16 19:10:04.628111912 +0000 UTC m=+1832.415676125"
Sep 16 19:10:04.978149 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:04.978102    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:05.983379 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:05.983343    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:06.603634 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:06.603595    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:06.604528 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:06.604505    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:10:16.013626 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:16.013590    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x"]
Sep 16 19:10:16.017539 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:16.017521    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x"
Sep 16 19:10:16.020739 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:16.020718    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nmr62\""
Sep 16 19:10:16.028744 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:16.028712    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x"]
Sep 16 19:10:16.102073 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:16.102028    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab3bad67-a05e-45d4-82e2-23fb72881680-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x\" (UID: \"ab3bad67-a05e-45d4-82e2-23fb72881680\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x"
Sep 16 19:10:16.202843 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:16.202785    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab3bad67-a05e-45d4-82e2-23fb72881680-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x\" (UID: \"ab3bad67-a05e-45d4-82e2-23fb72881680\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x"
Sep 16 19:10:16.203473 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:16.203450    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab3bad67-a05e-45d4-82e2-23fb72881680-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x\" (UID: \"ab3bad67-a05e-45d4-82e2-23fb72881680\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x"
Sep 16 19:10:16.330738 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:16.330646    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x"
Sep 16 19:10:16.641889 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:16.641850    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x"]
Sep 16 19:10:16.642075 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:10:16.642053    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab3bad67_a05e_45d4_82e2_23fb72881680.slice/crio-45c9de6b3167a52be534ac8f317c2a2a6cf09307eb7e36a77789c6aa45f214dc WatchSource:0}: Error finding container 45c9de6b3167a52be534ac8f317c2a2a6cf09307eb7e36a77789c6aa45f214dc: Status 404 returned error can't find the container with id 45c9de6b3167a52be534ac8f317c2a2a6cf09307eb7e36a77789c6aa45f214dc
Sep 16 19:10:16.690158 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:16.690073    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x" event={"ID":"ab3bad67-a05e-45d4-82e2-23fb72881680","Type":"ContainerStarted","Data":"45c9de6b3167a52be534ac8f317c2a2a6cf09307eb7e36a77789c6aa45f214dc"}
Sep 16 19:10:17.701612 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:17.701576    2578 generic.go:356] "Generic (PLEG): container finished" podID="ab3bad67-a05e-45d4-82e2-23fb72881680" containerID="93eb13238043376e5d2666919119dbb2b95ca6506385d0c2598437e381583b66" exitCode=0
Sep 16 19:10:17.701994 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:17.701656    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x" event={"ID":"ab3bad67-a05e-45d4-82e2-23fb72881680","Type":"ContainerDied","Data":"93eb13238043376e5d2666919119dbb2b95ca6506385d0c2598437e381583b66"}
Sep 16 19:10:18.853728 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:18.853704    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x"
Sep 16 19:10:18.934256 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:18.934218    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab3bad67-a05e-45d4-82e2-23fb72881680-maas-api-service-ca\") pod \"ab3bad67-a05e-45d4-82e2-23fb72881680\" (UID: \"ab3bad67-a05e-45d4-82e2-23fb72881680\") "
Sep 16 19:10:18.934676 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:18.934650    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab3bad67-a05e-45d4-82e2-23fb72881680-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ab3bad67-a05e-45d4-82e2-23fb72881680" (UID: "ab3bad67-a05e-45d4-82e2-23fb72881680"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:10:19.036154 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:19.036065    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab3bad67-a05e-45d4-82e2-23fb72881680-maas-api-service-ca\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:10:19.716691 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:19.716664    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x"
Sep 16 19:10:19.716855 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:19.716666    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x" event={"ID":"ab3bad67-a05e-45d4-82e2-23fb72881680","Type":"ContainerDied","Data":"45c9de6b3167a52be534ac8f317c2a2a6cf09307eb7e36a77789c6aa45f214dc"}
Sep 16 19:10:19.716855 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:19.716784    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45c9de6b3167a52be534ac8f317c2a2a6cf09307eb7e36a77789c6aa45f214dc"
Sep 16 19:10:20.914913 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:20.914877    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"]
Sep 16 19:10:23.744914 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:23.744825    2578 generic.go:356] "Generic (PLEG): container finished" podID="8098ba1b-391e-4688-98a2-f7ad0a7a72d8" containerID="1e0ec0656d44329d55c323579124b2750e5e3eddf75bdb991892470caa5cf049" exitCode=137
Sep 16 19:10:23.744914 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:23.744897    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr" event={"ID":"8098ba1b-391e-4688-98a2-f7ad0a7a72d8","Type":"ContainerDied","Data":"1e0ec0656d44329d55c323579124b2750e5e3eddf75bdb991892470caa5cf049"}
Sep 16 19:10:23.745380 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:23.745315    2578 scope.go:117] "RemoveContainer" containerID="1e0ec0656d44329d55c323579124b2750e5e3eddf75bdb991892470caa5cf049"
Sep 16 19:10:24.754269 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:24.754235    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr" event={"ID":"8098ba1b-391e-4688-98a2-f7ad0a7a72d8","Type":"ContainerStarted","Data":"3edf775a8cd20833d5bb1b45528aa86070f027ab05fbaa961eb1b0b0c0086666"}
Sep 16 19:10:24.754724 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:24.754448    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"
Sep 16 19:10:27.267739 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.267702    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"]
Sep 16 19:10:27.268337 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.268318    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab3bad67-a05e-45d4-82e2-23fb72881680" containerName="revoke-keys"
Sep 16 19:10:27.268337 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.268339    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab3bad67-a05e-45d4-82e2-23fb72881680" containerName="revoke-keys"
Sep 16 19:10:27.268469 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.268431    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab3bad67-a05e-45d4-82e2-23fb72881680" containerName="revoke-keys"
Sep 16 19:10:27.271750 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.271727    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.274836 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.274816    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-b04e50-openshift-default-dockercfg-r2kb6\""
Sep 16 19:10:27.294885 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.294844    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"]
Sep 16 19:10:27.429838 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.429802    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-credential-socket\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.430059 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.429882    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-workload-certs\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.430059 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.429934    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-workload-socket\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.430059 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.430044    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/376e752f-e054-4b59-8249-55e1112454e8-istio-podinfo\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.430264 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.430072    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v97bt\" (UniqueName: \"kubernetes.io/projected/376e752f-e054-4b59-8249-55e1112454e8-kube-api-access-v97bt\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.430264 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.430097    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/376e752f-e054-4b59-8249-55e1112454e8-istio-token\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.430264 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.430121    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-istio-data\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.430264 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.430242    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/376e752f-e054-4b59-8249-55e1112454e8-istiod-ca-cert\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.430392 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.430303    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-istio-envoy\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.531406 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.531304    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-credential-socket\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.531406 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.531376    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-workload-certs\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.531648 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.531419    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-workload-socket\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.531648 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.531486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/376e752f-e054-4b59-8249-55e1112454e8-istio-podinfo\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.531648 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.531512    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v97bt\" (UniqueName: \"kubernetes.io/projected/376e752f-e054-4b59-8249-55e1112454e8-kube-api-access-v97bt\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.531648 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.531537    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/376e752f-e054-4b59-8249-55e1112454e8-istio-token\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.531648 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.531560    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-istio-data\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.531648 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.531606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/376e752f-e054-4b59-8249-55e1112454e8-istiod-ca-cert\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.531911 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.531656    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-istio-envoy\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.531911 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.531773    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-credential-socket\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.532056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.532034    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-workload-socket\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.532253 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.532082    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-workload-certs\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.532253 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.532156    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-istio-data\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.532584 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.532563    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/376e752f-e054-4b59-8249-55e1112454e8-istiod-ca-cert\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.534042 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.534019    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-istio-envoy\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.534152 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.534134    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/376e752f-e054-4b59-8249-55e1112454e8-istio-podinfo\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.552393 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.552361    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/376e752f-e054-4b59-8249-55e1112454e8-istio-token\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.553337 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.553318    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v97bt\" (UniqueName: \"kubernetes.io/projected/376e752f-e054-4b59-8249-55e1112454e8-kube-api-access-v97bt\") pod \"e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") " pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.583302 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.583269    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:27.875255 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.875221    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"]
Sep 16 19:10:27.876334 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:10:27.876308    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod376e752f_e054_4b59_8249_55e1112454e8.slice/crio-65adecc427e7e1f64c83409ce1cfafa8ee7d63ec435afa9a0457312202724c92 WatchSource:0}: Error finding container 65adecc427e7e1f64c83409ce1cfafa8ee7d63ec435afa9a0457312202724c92: Status 404 returned error can't find the container with id 65adecc427e7e1f64c83409ce1cfafa8ee7d63ec435afa9a0457312202724c92
Sep 16 19:10:27.878974 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.878940    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:10:27.879046 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.879013    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:10:27.879046 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:27.879041    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:10:28.783200 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:28.783114    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq" event={"ID":"376e752f-e054-4b59-8249-55e1112454e8","Type":"ContainerStarted","Data":"956971b39692e07e9c296cb531c6e983a23c107b6575a1c26aa41b9b8b0df5ba"}
Sep 16 19:10:28.783564 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:28.783215    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq" event={"ID":"376e752f-e054-4b59-8249-55e1112454e8","Type":"ContainerStarted","Data":"65adecc427e7e1f64c83409ce1cfafa8ee7d63ec435afa9a0457312202724c92"}
Sep 16 19:10:28.821135 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:28.821079    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq" podStartSLOduration=1.821058582 podStartE2EDuration="1.821058582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:10:28.820459611 +0000 UTC m=+1856.608023785" watchObservedRunningTime="2026-09-16 19:10:28.821058582 +0000 UTC m=+1856.608622797"
Sep 16 19:10:29.583500 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:29.583463    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:29.588700 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:29.588673    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:29.790568 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:29.790532    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:29.791565 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:29.791546    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:10:33.088557 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:33.088515    2578 scope.go:117] "RemoveContainer" containerID="2c9acf783e49ccd60c38162d149bfd4bb531e31b9687b72fde17ac94720b0d6c"
Sep 16 19:10:33.098438 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:33.098418    2578 scope.go:117] "RemoveContainer" containerID="49c98e611129c28b5cab23c27b5cd22f2117195772a31c71b0110c000b3e082f"
Sep 16 19:10:35.762665 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:35.762633    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"
Sep 16 19:10:35.935346 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:35.935283    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2" podUID="9855a54a-d7e8-42b1-aed2-a2ed874b152a" containerName="main" containerID="cri-o://0de98ec83b56be4ecb4a2d440b0046917bf2727e737258c33bb3c2ed1fe940eb" gracePeriod=45
Sep 16 19:10:36.196102 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.196078    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:10:36.234364 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.234329    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-tmp-dir\") pod \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") "
Sep 16 19:10:36.234364 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.234366    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-home\") pod \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") "
Sep 16 19:10:36.234608 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.234414    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rtnl\" (UniqueName: \"kubernetes.io/projected/9855a54a-d7e8-42b1-aed2-a2ed874b152a-kube-api-access-9rtnl\") pod \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") "
Sep 16 19:10:36.234608 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.234452    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9855a54a-d7e8-42b1-aed2-a2ed874b152a-tls-certs\") pod \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") "
Sep 16 19:10:36.234608 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.234501    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-model-cache\") pod \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") "
Sep 16 19:10:36.234608 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.234573    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-dshm\") pod \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\" (UID: \"9855a54a-d7e8-42b1-aed2-a2ed874b152a\") "
Sep 16 19:10:36.234847 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.234800    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-model-cache" (OuterVolumeSpecName: "model-cache") pod "9855a54a-d7e8-42b1-aed2-a2ed874b152a" (UID: "9855a54a-d7e8-42b1-aed2-a2ed874b152a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:10:36.234910 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.234888    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9855a54a-d7e8-42b1-aed2-a2ed874b152a" (UID: "9855a54a-d7e8-42b1-aed2-a2ed874b152a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:10:36.235035 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.235011    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-tmp-dir\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:10:36.235035 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.235017    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-home" (OuterVolumeSpecName: "home") pod "9855a54a-d7e8-42b1-aed2-a2ed874b152a" (UID: "9855a54a-d7e8-42b1-aed2-a2ed874b152a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:10:36.235200 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.235043    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-model-cache\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:10:36.237394 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.237357    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9855a54a-d7e8-42b1-aed2-a2ed874b152a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9855a54a-d7e8-42b1-aed2-a2ed874b152a" (UID: "9855a54a-d7e8-42b1-aed2-a2ed874b152a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:10:36.237777 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.237755    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9855a54a-d7e8-42b1-aed2-a2ed874b152a-kube-api-access-9rtnl" (OuterVolumeSpecName: "kube-api-access-9rtnl") pod "9855a54a-d7e8-42b1-aed2-a2ed874b152a" (UID: "9855a54a-d7e8-42b1-aed2-a2ed874b152a"). InnerVolumeSpecName "kube-api-access-9rtnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:10:36.238082 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.238052    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-dshm" (OuterVolumeSpecName: "dshm") pod "9855a54a-d7e8-42b1-aed2-a2ed874b152a" (UID: "9855a54a-d7e8-42b1-aed2-a2ed874b152a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:10:36.336526 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.336422    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-home\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:10:36.336526 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.336458    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rtnl\" (UniqueName: \"kubernetes.io/projected/9855a54a-d7e8-42b1-aed2-a2ed874b152a-kube-api-access-9rtnl\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:10:36.336526 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.336470    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9855a54a-d7e8-42b1-aed2-a2ed874b152a-tls-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:10:36.336526 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.336481    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9855a54a-d7e8-42b1-aed2-a2ed874b152a-dshm\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:10:36.848332 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.848289    2578 generic.go:356] "Generic (PLEG): container finished" podID="9855a54a-d7e8-42b1-aed2-a2ed874b152a" containerID="0de98ec83b56be4ecb4a2d440b0046917bf2727e737258c33bb3c2ed1fe940eb" exitCode=0
Sep 16 19:10:36.848754 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.848339    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"
Sep 16 19:10:36.848754 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.848374    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2" event={"ID":"9855a54a-d7e8-42b1-aed2-a2ed874b152a","Type":"ContainerDied","Data":"0de98ec83b56be4ecb4a2d440b0046917bf2727e737258c33bb3c2ed1fe940eb"}
Sep 16 19:10:36.848754 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.848422    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2" event={"ID":"9855a54a-d7e8-42b1-aed2-a2ed874b152a","Type":"ContainerDied","Data":"2a19ec597e5458c5a8a17c42bf930d8a682c515f05b79840a6ccd512429d0152"}
Sep 16 19:10:36.848754 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.848439    2578 scope.go:117] "RemoveContainer" containerID="0de98ec83b56be4ecb4a2d440b0046917bf2727e737258c33bb3c2ed1fe940eb"
Sep 16 19:10:36.858302 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.858283    2578 scope.go:117] "RemoveContainer" containerID="0de98ec83b56be4ecb4a2d440b0046917bf2727e737258c33bb3c2ed1fe940eb"
Sep 16 19:10:36.858597 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:10:36.858578    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0de98ec83b56be4ecb4a2d440b0046917bf2727e737258c33bb3c2ed1fe940eb\": container with ID starting with 0de98ec83b56be4ecb4a2d440b0046917bf2727e737258c33bb3c2ed1fe940eb not found: ID does not exist" containerID="0de98ec83b56be4ecb4a2d440b0046917bf2727e737258c33bb3c2ed1fe940eb"
Sep 16 19:10:36.858686 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.858605    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0de98ec83b56be4ecb4a2d440b0046917bf2727e737258c33bb3c2ed1fe940eb"} err="failed to get container status \"0de98ec83b56be4ecb4a2d440b0046917bf2727e737258c33bb3c2ed1fe940eb\": rpc error: code = NotFound desc = could not find container \"0de98ec83b56be4ecb4a2d440b0046917bf2727e737258c33bb3c2ed1fe940eb\": container with ID starting with 0de98ec83b56be4ecb4a2d440b0046917bf2727e737258c33bb3c2ed1fe940eb not found: ID does not exist"
Sep 16 19:10:36.879738 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.879696    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"]
Sep 16 19:10:36.888083 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:36.888053    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-b1857c/embedding-w0-b1857c-kserve-569cbdf44d-5h2b2"]
Sep 16 19:10:38.710539 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:38.710504    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9855a54a-d7e8-42b1-aed2-a2ed874b152a" path="/var/lib/kubelet/pods/9855a54a-d7e8-42b1-aed2-a2ed874b152a/volumes"
Sep 16 19:10:41.244696 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:41.244659    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x"]
Sep 16 19:10:41.247817 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:41.247785    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-b1857c-8qc9x"]
Sep 16 19:10:42.710649 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:42.710612    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab3bad67-a05e-45d4-82e2-23fb72881680" path="/var/lib/kubelet/pods/ab3bad67-a05e-45d4-82e2-23fb72881680/volumes"
Sep 16 19:10:49.158452 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:49.158412    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"]
Sep 16 19:10:49.158950 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:49.158609    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5" podUID="c292ec82-49cf-4c8d-9072-6eff2febbe63" containerName="istio-proxy" containerID="cri-o://18eb0365636b2279141ad6369a0d31053f9e1138ab307187142be0d3dcd2b77a" gracePeriod=30
Sep 16 19:10:54.412241 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.412213    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 19:10:54.540575 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.540478    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dz2xl\" (UniqueName: \"kubernetes.io/projected/c292ec82-49cf-4c8d-9072-6eff2febbe63-kube-api-access-dz2xl\") pod \"c292ec82-49cf-4c8d-9072-6eff2febbe63\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") "
Sep 16 19:10:54.540575 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.540522    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c292ec82-49cf-4c8d-9072-6eff2febbe63-istiod-ca-cert\") pod \"c292ec82-49cf-4c8d-9072-6eff2febbe63\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") "
Sep 16 19:10:54.540575 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.540559    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-credential-socket\") pod \"c292ec82-49cf-4c8d-9072-6eff2febbe63\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") "
Sep 16 19:10:54.540912 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.540711    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-data\") pod \"c292ec82-49cf-4c8d-9072-6eff2febbe63\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") "
Sep 16 19:10:54.540912 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.540864    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c292ec82-49cf-4c8d-9072-6eff2febbe63" (UID: "c292ec82-49cf-4c8d-9072-6eff2febbe63"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:10:54.541039 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.540910    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-podinfo\") pod \"c292ec82-49cf-4c8d-9072-6eff2febbe63\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") "
Sep 16 19:10:54.541039 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.540953    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-workload-certs\") pod \"c292ec82-49cf-4c8d-9072-6eff2febbe63\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") "
Sep 16 19:10:54.541039 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.540983    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-token\") pod \"c292ec82-49cf-4c8d-9072-6eff2febbe63\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") "
Sep 16 19:10:54.541039 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.540950    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c292ec82-49cf-4c8d-9072-6eff2febbe63-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c292ec82-49cf-4c8d-9072-6eff2febbe63" (UID: "c292ec82-49cf-4c8d-9072-6eff2febbe63"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:10:54.541335 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.541074    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-workload-socket\") pod \"c292ec82-49cf-4c8d-9072-6eff2febbe63\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") "
Sep 16 19:10:54.541335 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.541120    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-envoy\") pod \"c292ec82-49cf-4c8d-9072-6eff2febbe63\" (UID: \"c292ec82-49cf-4c8d-9072-6eff2febbe63\") "
Sep 16 19:10:54.541335 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.541143    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-data" (OuterVolumeSpecName: "istio-data") pod "c292ec82-49cf-4c8d-9072-6eff2febbe63" (UID: "c292ec82-49cf-4c8d-9072-6eff2febbe63"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:10:54.541335 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.541203    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c292ec82-49cf-4c8d-9072-6eff2febbe63" (UID: "c292ec82-49cf-4c8d-9072-6eff2febbe63"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:10:54.541561 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.541371    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c292ec82-49cf-4c8d-9072-6eff2febbe63" (UID: "c292ec82-49cf-4c8d-9072-6eff2febbe63"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:10:54.541742 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.541708    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:10:54.541742 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.541733    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c292ec82-49cf-4c8d-9072-6eff2febbe63-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:10:54.541958 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.541747    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:10:54.541958 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.541759    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:10:54.541958 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.541772    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:10:54.544011 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.543980    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c292ec82-49cf-4c8d-9072-6eff2febbe63-kube-api-access-dz2xl" (OuterVolumeSpecName: "kube-api-access-dz2xl") pod "c292ec82-49cf-4c8d-9072-6eff2febbe63" (UID: "c292ec82-49cf-4c8d-9072-6eff2febbe63"). InnerVolumeSpecName "kube-api-access-dz2xl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:10:54.544011 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.543987    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c292ec82-49cf-4c8d-9072-6eff2febbe63" (UID: "c292ec82-49cf-4c8d-9072-6eff2febbe63"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:10:54.544011 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.544007    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-token" (OuterVolumeSpecName: "istio-token") pod "c292ec82-49cf-4c8d-9072-6eff2febbe63" (UID: "c292ec82-49cf-4c8d-9072-6eff2febbe63"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:10:54.544287 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.544051    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c292ec82-49cf-4c8d-9072-6eff2febbe63" (UID: "c292ec82-49cf-4c8d-9072-6eff2febbe63"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:10:54.642861 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.642812    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dz2xl\" (UniqueName: \"kubernetes.io/projected/c292ec82-49cf-4c8d-9072-6eff2febbe63-kube-api-access-dz2xl\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:10:54.642861 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.642862    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:10:54.643080 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.642878    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:10:54.643080 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.642891    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c292ec82-49cf-4c8d-9072-6eff2febbe63-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:10:54.980569 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.980537    2578 generic.go:356] "Generic (PLEG): container finished" podID="c292ec82-49cf-4c8d-9072-6eff2febbe63" containerID="18eb0365636b2279141ad6369a0d31053f9e1138ab307187142be0d3dcd2b77a" exitCode=0
Sep 16 19:10:54.980761 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.980588    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"
Sep 16 19:10:54.980761 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.980630    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5" event={"ID":"c292ec82-49cf-4c8d-9072-6eff2febbe63","Type":"ContainerDied","Data":"18eb0365636b2279141ad6369a0d31053f9e1138ab307187142be0d3dcd2b77a"}
Sep 16 19:10:54.980761 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.980684    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5" event={"ID":"c292ec82-49cf-4c8d-9072-6eff2febbe63","Type":"ContainerDied","Data":"046b5ec07df9c4dbac322ed2e635b7e2d6119340072828f7ffb3c274d9fcba11"}
Sep 16 19:10:54.980761 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.980708    2578 scope.go:117] "RemoveContainer" containerID="18eb0365636b2279141ad6369a0d31053f9e1138ab307187142be0d3dcd2b77a"
Sep 16 19:10:54.990687 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:54.990670    2578 scope.go:117] "RemoveContainer" containerID="4ab56b9bb01ab732241beb530642cc88fb1785b3cc6454143bd6bbd770c7b213"
Sep 16 19:10:55.000803 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:55.000787    2578 scope.go:117] "RemoveContainer" containerID="18eb0365636b2279141ad6369a0d31053f9e1138ab307187142be0d3dcd2b77a"
Sep 16 19:10:55.001116 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:10:55.001098    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18eb0365636b2279141ad6369a0d31053f9e1138ab307187142be0d3dcd2b77a\": container with ID starting with 18eb0365636b2279141ad6369a0d31053f9e1138ab307187142be0d3dcd2b77a not found: ID does not exist" containerID="18eb0365636b2279141ad6369a0d31053f9e1138ab307187142be0d3dcd2b77a"
Sep 16 19:10:55.001222 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:55.001129    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18eb0365636b2279141ad6369a0d31053f9e1138ab307187142be0d3dcd2b77a"} err="failed to get container status \"18eb0365636b2279141ad6369a0d31053f9e1138ab307187142be0d3dcd2b77a\": rpc error: code = NotFound desc = could not find container \"18eb0365636b2279141ad6369a0d31053f9e1138ab307187142be0d3dcd2b77a\": container with ID starting with 18eb0365636b2279141ad6369a0d31053f9e1138ab307187142be0d3dcd2b77a not found: ID does not exist"
Sep 16 19:10:55.001222 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:55.001153    2578 scope.go:117] "RemoveContainer" containerID="4ab56b9bb01ab732241beb530642cc88fb1785b3cc6454143bd6bbd770c7b213"
Sep 16 19:10:55.001451 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:10:55.001431    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ab56b9bb01ab732241beb530642cc88fb1785b3cc6454143bd6bbd770c7b213\": container with ID starting with 4ab56b9bb01ab732241beb530642cc88fb1785b3cc6454143bd6bbd770c7b213 not found: ID does not exist" containerID="4ab56b9bb01ab732241beb530642cc88fb1785b3cc6454143bd6bbd770c7b213"
Sep 16 19:10:55.001494 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:55.001457    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ab56b9bb01ab732241beb530642cc88fb1785b3cc6454143bd6bbd770c7b213"} err="failed to get container status \"4ab56b9bb01ab732241beb530642cc88fb1785b3cc6454143bd6bbd770c7b213\": rpc error: code = NotFound desc = could not find container \"4ab56b9bb01ab732241beb530642cc88fb1785b3cc6454143bd6bbd770c7b213\": container with ID starting with 4ab56b9bb01ab732241beb530642cc88fb1785b3cc6454143bd6bbd770c7b213 not found: ID does not exist"
Sep 16 19:10:55.001943 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:55.001925    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"]
Sep 16 19:10:55.008675 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:55.008655    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1857c-openshift-default-5cfd78d748-cb2s5"]
Sep 16 19:10:56.713801 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:10:56.713762    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c292ec82-49cf-4c8d-9072-6eff2febbe63" path="/var/lib/kubelet/pods/c292ec82-49cf-4c8d-9072-6eff2febbe63/volumes"
Sep 16 19:11:06.080507 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:06.080410    2578 generic.go:356] "Generic (PLEG): container finished" podID="b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" containerID="628f7237b8a9620e297e36f36efda7b8ed303441efde0c04db991b5881f50450" exitCode=0
Sep 16 19:11:06.080507 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:06.080485    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz" event={"ID":"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6","Type":"ContainerDied","Data":"628f7237b8a9620e297e36f36efda7b8ed303441efde0c04db991b5881f50450"}
Sep 16 19:11:06.081070 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:06.080957    2578 scope.go:117] "RemoveContainer" containerID="628f7237b8a9620e297e36f36efda7b8ed303441efde0c04db991b5881f50450"
Sep 16 19:11:06.082359 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:06.082323    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:11:06.082488 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:06.082415    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:11:06.082488 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:06.082467    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:11:06.082760 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:06.082741    2578 generic.go:356] "Generic (PLEG): container finished" podID="376e752f-e054-4b59-8249-55e1112454e8" containerID="956971b39692e07e9c296cb531c6e983a23c107b6575a1c26aa41b9b8b0df5ba" exitCode=0
Sep 16 19:11:06.082853 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:06.082813    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq" event={"ID":"376e752f-e054-4b59-8249-55e1112454e8","Type":"ContainerDied","Data":"956971b39692e07e9c296cb531c6e983a23c107b6575a1c26aa41b9b8b0df5ba"}
Sep 16 19:11:06.083282 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:06.083261    2578 scope.go:117] "RemoveContainer" containerID="956971b39692e07e9c296cb531c6e983a23c107b6575a1c26aa41b9b8b0df5ba"
Sep 16 19:11:06.084465 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:06.084435    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:11:06.084566 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:06.084504    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:11:06.084566 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:06.084541    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:11:06.603534 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:06.603498    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:11:07.102184 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:07.102148    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz" event={"ID":"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6","Type":"ContainerStarted","Data":"dd105578a2035497c24dd7f23b13f0da269e92a259b6311835010986ff354358"}
Sep 16 19:11:07.104296 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:07.104271    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq" event={"ID":"376e752f-e054-4b59-8249-55e1112454e8","Type":"ContainerStarted","Data":"6f9505cf29a475a23cee7b254f13c43148ed3848d4d6abf26d814f032d9bad6c"}
Sep 16 19:11:07.584575 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:07.584539    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:11:07.584981 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:07.584959    2578 patch_prober.go:28] interesting pod/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Sep 16 19:11:07.585134 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:07.585110    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq" podUID="376e752f-e054-4b59-8249-55e1112454e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Sep 16 19:11:07.978744 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:07.978700    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:11:07.980050 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:07.980014    2578 patch_prober.go:28] interesting pod/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 16 19:11:07.980207 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:07.980088    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz" podUID="b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 16 19:11:08.583885 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:08.583838    2578 patch_prober.go:28] interesting pod/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Sep 16 19:11:08.584440 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:08.583929    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq" podUID="376e752f-e054-4b59-8249-55e1112454e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Sep 16 19:11:08.978708 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:08.978661    2578 patch_prober.go:28] interesting pod/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 16 19:11:08.978898 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:08.978731    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz" podUID="b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 16 19:11:09.588198 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:09.587588    2578 patch_prober.go:28] interesting pod/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Sep 16 19:11:09.588198 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:09.587654    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq" podUID="376e752f-e054-4b59-8249-55e1112454e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Sep 16 19:11:09.987197 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:09.986842    2578 patch_prober.go:28] interesting pod/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 16 19:11:09.987197 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:09.986907    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz" podUID="b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 16 19:11:10.584544 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:10.584494    2578 patch_prober.go:28] interesting pod/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Sep 16 19:11:10.584783 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:10.584587    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq" podUID="376e752f-e054-4b59-8249-55e1112454e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Sep 16 19:11:10.982059 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:10.981575    2578 patch_prober.go:28] interesting pod/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 16 19:11:10.982059 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:10.981660    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz" podUID="b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 16 19:11:11.583978 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:11.583941    2578 patch_prober.go:28] interesting pod/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Sep 16 19:11:11.584290 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:11.584258    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq" podUID="376e752f-e054-4b59-8249-55e1112454e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Sep 16 19:11:11.979144 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:11.978664    2578 patch_prober.go:28] interesting pod/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Sep 16 19:11:11.979144 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:11.978732    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz" podUID="b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Sep 16 19:11:12.652040 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:12.652006    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:11:12.652883 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:12.652844    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:11:12.655716 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:12.655696    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:11:12.983557 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:12.983415    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:11:12.986313 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:12.986275    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:11:12.991640 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:12.991604    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:11:18.887123 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:18.887082    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"]
Sep 16 19:11:18.887576 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:18.887334    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz" podUID="b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" containerName="istio-proxy" containerID="cri-o://dd105578a2035497c24dd7f23b13f0da269e92a259b6311835010986ff354358" gracePeriod=30
Sep 16 19:11:24.159855 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.159792    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:11:24.230843 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.230806    2578 generic.go:356] "Generic (PLEG): container finished" podID="b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" containerID="dd105578a2035497c24dd7f23b13f0da269e92a259b6311835010986ff354358" exitCode=0
Sep 16 19:11:24.231007 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.230861    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"
Sep 16 19:11:24.231007 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.230893    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz" event={"ID":"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6","Type":"ContainerDied","Data":"dd105578a2035497c24dd7f23b13f0da269e92a259b6311835010986ff354358"}
Sep 16 19:11:24.231007 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.230951    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz" event={"ID":"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6","Type":"ContainerDied","Data":"fedf3282ab8c71dbdc488614ddc57e2d27e9d0cd98b108ed82a4a4c7daa2c893"}
Sep 16 19:11:24.231007 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.230974    2578 scope.go:117] "RemoveContainer" containerID="dd105578a2035497c24dd7f23b13f0da269e92a259b6311835010986ff354358"
Sep 16 19:11:24.240412 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.240391    2578 scope.go:117] "RemoveContainer" containerID="628f7237b8a9620e297e36f36efda7b8ed303441efde0c04db991b5881f50450"
Sep 16 19:11:24.249964 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.249944    2578 scope.go:117] "RemoveContainer" containerID="dd105578a2035497c24dd7f23b13f0da269e92a259b6311835010986ff354358"
Sep 16 19:11:24.250317 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:11:24.250291    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd105578a2035497c24dd7f23b13f0da269e92a259b6311835010986ff354358\": container with ID starting with dd105578a2035497c24dd7f23b13f0da269e92a259b6311835010986ff354358 not found: ID does not exist" containerID="dd105578a2035497c24dd7f23b13f0da269e92a259b6311835010986ff354358"
Sep 16 19:11:24.250394 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.250330    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd105578a2035497c24dd7f23b13f0da269e92a259b6311835010986ff354358"} err="failed to get container status \"dd105578a2035497c24dd7f23b13f0da269e92a259b6311835010986ff354358\": rpc error: code = NotFound desc = could not find container \"dd105578a2035497c24dd7f23b13f0da269e92a259b6311835010986ff354358\": container with ID starting with dd105578a2035497c24dd7f23b13f0da269e92a259b6311835010986ff354358 not found: ID does not exist"
Sep 16 19:11:24.250394 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.250354    2578 scope.go:117] "RemoveContainer" containerID="628f7237b8a9620e297e36f36efda7b8ed303441efde0c04db991b5881f50450"
Sep 16 19:11:24.250623 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:11:24.250604    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"628f7237b8a9620e297e36f36efda7b8ed303441efde0c04db991b5881f50450\": container with ID starting with 628f7237b8a9620e297e36f36efda7b8ed303441efde0c04db991b5881f50450 not found: ID does not exist" containerID="628f7237b8a9620e297e36f36efda7b8ed303441efde0c04db991b5881f50450"
Sep 16 19:11:24.250672 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.250629    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"628f7237b8a9620e297e36f36efda7b8ed303441efde0c04db991b5881f50450"} err="failed to get container status \"628f7237b8a9620e297e36f36efda7b8ed303441efde0c04db991b5881f50450\": rpc error: code = NotFound desc = could not find container \"628f7237b8a9620e297e36f36efda7b8ed303441efde0c04db991b5881f50450\": container with ID starting with 628f7237b8a9620e297e36f36efda7b8ed303441efde0c04db991b5881f50450 not found: ID does not exist"
Sep 16 19:11:24.303054 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.303018    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-workload-socket\") pod \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") "
Sep 16 19:11:24.303286 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.303114    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-envoy\") pod \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") "
Sep 16 19:11:24.303286 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.303152    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-credential-socket\") pod \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") "
Sep 16 19:11:24.303286 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.303253    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-workload-certs\") pod \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") "
Sep 16 19:11:24.303286 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.303263    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" (UID: "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:11:24.303517 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.303302    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istiod-ca-cert\") pod \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") "
Sep 16 19:11:24.303517 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.303354    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wqcs\" (UniqueName: \"kubernetes.io/projected/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-kube-api-access-4wqcs\") pod \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") "
Sep 16 19:11:24.303517 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.303381    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-podinfo\") pod \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") "
Sep 16 19:11:24.303517 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.303419    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-token\") pod \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") "
Sep 16 19:11:24.303517 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.303449    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" (UID: "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:11:24.303517 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.303461    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-data\") pod \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\" (UID: \"b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6\") "
Sep 16 19:11:24.303517 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.303461    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" (UID: "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:11:24.303891 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.303691    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" (UID: "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:11:24.303891 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.303712    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-data" (OuterVolumeSpecName: "istio-data") pod "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" (UID: "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:11:24.303995 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.303970    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:24.304056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.303999    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:24.304056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.304011    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:24.304056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.304023    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:24.304056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.304038    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:24.305504 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.305466    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" (UID: "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:11:24.305815 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.305784    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" (UID: "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:11:24.305815 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.305791    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-kube-api-access-4wqcs" (OuterVolumeSpecName: "kube-api-access-4wqcs") pod "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" (UID: "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6"). InnerVolumeSpecName "kube-api-access-4wqcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:11:24.305960 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.305845    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" (UID: "b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:11:24.405195 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.405133    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wqcs\" (UniqueName: \"kubernetes.io/projected/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-kube-api-access-4wqcs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:24.405195 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.405163    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:24.405195 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.405186    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:24.405195 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.405196    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:24.561339 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.561305    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"]
Sep 16 19:11:24.567119 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.567091    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a46230-openshift-default-56b789c668-qs2kz"]
Sep 16 19:11:24.711863 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:24.711827    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" path="/var/lib/kubelet/pods/b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6/volumes"
Sep 16 19:11:33.129769 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:33.129735    2578 scope.go:117] "RemoveContainer" containerID="b5e5e73875ddfe40c77a264043d689cb019b80bba4dee24a72f4bcf79259c141"
Sep 16 19:11:33.139793 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:33.139765    2578 scope.go:117] "RemoveContainer" containerID="3d60d4475f450a10262d8d2acfeab39227efe1eb5b0b0dc757a140ccf8eaa84e"
Sep 16 19:11:43.629509 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:43.629475    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"]
Sep 16 19:11:43.630010 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:43.629709    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq" podUID="376e752f-e054-4b59-8249-55e1112454e8" containerName="istio-proxy" containerID="cri-o://6f9505cf29a475a23cee7b254f13c43148ed3848d4d6abf26d814f032d9bad6c" gracePeriod=30
Sep 16 19:11:48.901747 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.901686    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:11:48.966237 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.966199    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-credential-socket\") pod \"376e752f-e054-4b59-8249-55e1112454e8\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") "
Sep 16 19:11:48.966405 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.966246    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/376e752f-e054-4b59-8249-55e1112454e8-istio-podinfo\") pod \"376e752f-e054-4b59-8249-55e1112454e8\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") "
Sep 16 19:11:48.966405 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.966265    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-istio-data\") pod \"376e752f-e054-4b59-8249-55e1112454e8\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") "
Sep 16 19:11:48.966405 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.966301    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-istio-envoy\") pod \"376e752f-e054-4b59-8249-55e1112454e8\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") "
Sep 16 19:11:48.966599 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.966431    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/376e752f-e054-4b59-8249-55e1112454e8-istiod-ca-cert\") pod \"376e752f-e054-4b59-8249-55e1112454e8\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") "
Sep 16 19:11:48.966599 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.966536    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v97bt\" (UniqueName: \"kubernetes.io/projected/376e752f-e054-4b59-8249-55e1112454e8-kube-api-access-v97bt\") pod \"376e752f-e054-4b59-8249-55e1112454e8\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") "
Sep 16 19:11:48.966599 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.966561    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "376e752f-e054-4b59-8249-55e1112454e8" (UID: "376e752f-e054-4b59-8249-55e1112454e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:11:48.966853 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.966661    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/376e752f-e054-4b59-8249-55e1112454e8-istio-token\") pod \"376e752f-e054-4b59-8249-55e1112454e8\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") "
Sep 16 19:11:48.966853 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.966669    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "376e752f-e054-4b59-8249-55e1112454e8" (UID: "376e752f-e054-4b59-8249-55e1112454e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:11:48.966853 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.966716    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-workload-certs\") pod \"376e752f-e054-4b59-8249-55e1112454e8\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") "
Sep 16 19:11:48.966853 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.966742    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-workload-socket\") pod \"376e752f-e054-4b59-8249-55e1112454e8\" (UID: \"376e752f-e054-4b59-8249-55e1112454e8\") "
Sep 16 19:11:48.967075 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.966905    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/376e752f-e054-4b59-8249-55e1112454e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "376e752f-e054-4b59-8249-55e1112454e8" (UID: "376e752f-e054-4b59-8249-55e1112454e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:11:48.967075 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.967000    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "376e752f-e054-4b59-8249-55e1112454e8" (UID: "376e752f-e054-4b59-8249-55e1112454e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:11:48.967270 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.967254    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:48.967333 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.967276    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:48.967333 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.967292    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:48.967333 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.967305    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/376e752f-e054-4b59-8249-55e1112454e8-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:48.967333 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.967307    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "376e752f-e054-4b59-8249-55e1112454e8" (UID: "376e752f-e054-4b59-8249-55e1112454e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:11:48.968714 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.968687    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "376e752f-e054-4b59-8249-55e1112454e8" (UID: "376e752f-e054-4b59-8249-55e1112454e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:11:48.968840 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.968816    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/376e752f-e054-4b59-8249-55e1112454e8-kube-api-access-v97bt" (OuterVolumeSpecName: "kube-api-access-v97bt") pod "376e752f-e054-4b59-8249-55e1112454e8" (UID: "376e752f-e054-4b59-8249-55e1112454e8"). InnerVolumeSpecName "kube-api-access-v97bt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:11:48.968938 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.968919    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/376e752f-e054-4b59-8249-55e1112454e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "376e752f-e054-4b59-8249-55e1112454e8" (UID: "376e752f-e054-4b59-8249-55e1112454e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:11:48.969189 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:48.969151    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/376e752f-e054-4b59-8249-55e1112454e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "376e752f-e054-4b59-8249-55e1112454e8" (UID: "376e752f-e054-4b59-8249-55e1112454e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:11:49.068014 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.067978    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/376e752f-e054-4b59-8249-55e1112454e8-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:49.068014 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.068009    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:49.068014 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.068019    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v97bt\" (UniqueName: \"kubernetes.io/projected/376e752f-e054-4b59-8249-55e1112454e8-kube-api-access-v97bt\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:49.068294 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.068029    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/376e752f-e054-4b59-8249-55e1112454e8-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:49.068294 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.068038    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/376e752f-e054-4b59-8249-55e1112454e8-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:11:49.400447 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.400416    2578 generic.go:356] "Generic (PLEG): container finished" podID="376e752f-e054-4b59-8249-55e1112454e8" containerID="6f9505cf29a475a23cee7b254f13c43148ed3848d4d6abf26d814f032d9bad6c" exitCode=0
Sep 16 19:11:49.400619 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.400462    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"
Sep 16 19:11:49.400619 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.400498    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq" event={"ID":"376e752f-e054-4b59-8249-55e1112454e8","Type":"ContainerDied","Data":"6f9505cf29a475a23cee7b254f13c43148ed3848d4d6abf26d814f032d9bad6c"}
Sep 16 19:11:49.400619 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.400584    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq" event={"ID":"376e752f-e054-4b59-8249-55e1112454e8","Type":"ContainerDied","Data":"65adecc427e7e1f64c83409ce1cfafa8ee7d63ec435afa9a0457312202724c92"}
Sep 16 19:11:49.400619 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.400609    2578 scope.go:117] "RemoveContainer" containerID="6f9505cf29a475a23cee7b254f13c43148ed3848d4d6abf26d814f032d9bad6c"
Sep 16 19:11:49.411613 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.411597    2578 scope.go:117] "RemoveContainer" containerID="956971b39692e07e9c296cb531c6e983a23c107b6575a1c26aa41b9b8b0df5ba"
Sep 16 19:11:49.424664 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.424640    2578 scope.go:117] "RemoveContainer" containerID="6f9505cf29a475a23cee7b254f13c43148ed3848d4d6abf26d814f032d9bad6c"
Sep 16 19:11:49.424975 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:11:49.424954    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f9505cf29a475a23cee7b254f13c43148ed3848d4d6abf26d814f032d9bad6c\": container with ID starting with 6f9505cf29a475a23cee7b254f13c43148ed3848d4d6abf26d814f032d9bad6c not found: ID does not exist" containerID="6f9505cf29a475a23cee7b254f13c43148ed3848d4d6abf26d814f032d9bad6c"
Sep 16 19:11:49.425024 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.424988    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f9505cf29a475a23cee7b254f13c43148ed3848d4d6abf26d814f032d9bad6c"} err="failed to get container status \"6f9505cf29a475a23cee7b254f13c43148ed3848d4d6abf26d814f032d9bad6c\": rpc error: code = NotFound desc = could not find container \"6f9505cf29a475a23cee7b254f13c43148ed3848d4d6abf26d814f032d9bad6c\": container with ID starting with 6f9505cf29a475a23cee7b254f13c43148ed3848d4d6abf26d814f032d9bad6c not found: ID does not exist"
Sep 16 19:11:49.425024 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.425006    2578 scope.go:117] "RemoveContainer" containerID="956971b39692e07e9c296cb531c6e983a23c107b6575a1c26aa41b9b8b0df5ba"
Sep 16 19:11:49.425293 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:11:49.425278    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"956971b39692e07e9c296cb531c6e983a23c107b6575a1c26aa41b9b8b0df5ba\": container with ID starting with 956971b39692e07e9c296cb531c6e983a23c107b6575a1c26aa41b9b8b0df5ba not found: ID does not exist" containerID="956971b39692e07e9c296cb531c6e983a23c107b6575a1c26aa41b9b8b0df5ba"
Sep 16 19:11:49.425343 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.425298    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"956971b39692e07e9c296cb531c6e983a23c107b6575a1c26aa41b9b8b0df5ba"} err="failed to get container status \"956971b39692e07e9c296cb531c6e983a23c107b6575a1c26aa41b9b8b0df5ba\": rpc error: code = NotFound desc = could not find container \"956971b39692e07e9c296cb531c6e983a23c107b6575a1c26aa41b9b8b0df5ba\": container with ID starting with 956971b39692e07e9c296cb531c6e983a23c107b6575a1c26aa41b9b8b0df5ba not found: ID does not exist"
Sep 16 19:11:49.432625 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.432603    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"]
Sep 16 19:11:49.440662 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:49.440639    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-b04e50-openshift-default-586b4df5cd-957gq"]
Sep 16 19:11:50.710026 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:11:50.709988    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="376e752f-e054-4b59-8249-55e1112454e8" path="/var/lib/kubelet/pods/376e752f-e054-4b59-8249-55e1112454e8/volumes"
Sep 16 19:12:02.981076 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:02.981046    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"]
Sep 16 19:12:02.981496 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:02.981272    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7" podUID="b163981d-51c3-4f96-9597-228867b865e5" containerName="istio-proxy" containerID="cri-o://ea2c8bb6092a65469cd9f4f4f69fe129da3eb452bda67a072956de35785b8d2b" gracePeriod=30
Sep 16 19:12:03.694482 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:03.694439    2578 patch_prober.go:28] interesting pod/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 19:12:03.694683 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:03.694522    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7" podUID="b163981d-51c3-4f96-9597-228867b865e5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:12:08.239810 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.239779    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:12:08.272777 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.272702    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-credential-socket\") pod \"b163981d-51c3-4f96-9597-228867b865e5\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") "
Sep 16 19:12:08.272777 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.272744    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b163981d-51c3-4f96-9597-228867b865e5-istio-token\") pod \"b163981d-51c3-4f96-9597-228867b865e5\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") "
Sep 16 19:12:08.272991 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.272787    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-workload-socket\") pod \"b163981d-51c3-4f96-9597-228867b865e5\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") "
Sep 16 19:12:08.272991 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.272803    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b163981d-51c3-4f96-9597-228867b865e5-istiod-ca-cert\") pod \"b163981d-51c3-4f96-9597-228867b865e5\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") "
Sep 16 19:12:08.272991 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.272829    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-istio-data\") pod \"b163981d-51c3-4f96-9597-228867b865e5\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") "
Sep 16 19:12:08.272991 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.272853    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-istio-envoy\") pod \"b163981d-51c3-4f96-9597-228867b865e5\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") "
Sep 16 19:12:08.272991 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.272926    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzjp4\" (UniqueName: \"kubernetes.io/projected/b163981d-51c3-4f96-9597-228867b865e5-kube-api-access-kzjp4\") pod \"b163981d-51c3-4f96-9597-228867b865e5\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") "
Sep 16 19:12:08.273291 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.273001    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b163981d-51c3-4f96-9597-228867b865e5" (UID: "b163981d-51c3-4f96-9597-228867b865e5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:12:08.273291 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.273014    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b163981d-51c3-4f96-9597-228867b865e5-istio-podinfo\") pod \"b163981d-51c3-4f96-9597-228867b865e5\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") "
Sep 16 19:12:08.273291 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.273110    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-workload-certs\") pod \"b163981d-51c3-4f96-9597-228867b865e5\" (UID: \"b163981d-51c3-4f96-9597-228867b865e5\") "
Sep 16 19:12:08.273612 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.273558    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:08.273612 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.273108    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b163981d-51c3-4f96-9597-228867b865e5" (UID: "b163981d-51c3-4f96-9597-228867b865e5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:12:08.273612 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.273128    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-istio-data" (OuterVolumeSpecName: "istio-data") pod "b163981d-51c3-4f96-9597-228867b865e5" (UID: "b163981d-51c3-4f96-9597-228867b865e5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:12:08.273612 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.273198    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b163981d-51c3-4f96-9597-228867b865e5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b163981d-51c3-4f96-9597-228867b865e5" (UID: "b163981d-51c3-4f96-9597-228867b865e5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:12:08.273902 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.273662    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b163981d-51c3-4f96-9597-228867b865e5" (UID: "b163981d-51c3-4f96-9597-228867b865e5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:12:08.275238 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.275214    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b163981d-51c3-4f96-9597-228867b865e5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b163981d-51c3-4f96-9597-228867b865e5" (UID: "b163981d-51c3-4f96-9597-228867b865e5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:12:08.275466 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.275444    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b163981d-51c3-4f96-9597-228867b865e5-kube-api-access-kzjp4" (OuterVolumeSpecName: "kube-api-access-kzjp4") pod "b163981d-51c3-4f96-9597-228867b865e5" (UID: "b163981d-51c3-4f96-9597-228867b865e5"). InnerVolumeSpecName "kube-api-access-kzjp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:12:08.275695 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.275670    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b163981d-51c3-4f96-9597-228867b865e5" (UID: "b163981d-51c3-4f96-9597-228867b865e5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:12:08.275695 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.275680    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b163981d-51c3-4f96-9597-228867b865e5-istio-token" (OuterVolumeSpecName: "istio-token") pod "b163981d-51c3-4f96-9597-228867b865e5" (UID: "b163981d-51c3-4f96-9597-228867b865e5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:12:08.374113 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.374076    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:08.374113 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.374107    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b163981d-51c3-4f96-9597-228867b865e5-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:08.374113 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.374119    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:08.374432 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.374127    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:08.374432 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.374135    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzjp4\" (UniqueName: \"kubernetes.io/projected/b163981d-51c3-4f96-9597-228867b865e5-kube-api-access-kzjp4\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:08.374432 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.374143    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b163981d-51c3-4f96-9597-228867b865e5-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:08.374432 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.374151    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b163981d-51c3-4f96-9597-228867b865e5-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:08.374432 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.374161    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b163981d-51c3-4f96-9597-228867b865e5-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:08.542948 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.542850    2578 generic.go:356] "Generic (PLEG): container finished" podID="b163981d-51c3-4f96-9597-228867b865e5" containerID="ea2c8bb6092a65469cd9f4f4f69fe129da3eb452bda67a072956de35785b8d2b" exitCode=0
Sep 16 19:12:08.542948 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.542915    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"
Sep 16 19:12:08.543214 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.542981    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7" event={"ID":"b163981d-51c3-4f96-9597-228867b865e5","Type":"ContainerDied","Data":"ea2c8bb6092a65469cd9f4f4f69fe129da3eb452bda67a072956de35785b8d2b"}
Sep 16 19:12:08.543214 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.543037    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7" event={"ID":"b163981d-51c3-4f96-9597-228867b865e5","Type":"ContainerDied","Data":"aa7ccd03d9557efc88cd9e3edfdcf0cc86cc197d1f7b1c8d1bd2bdc338e61249"}
Sep 16 19:12:08.543214 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.543081    2578 scope.go:117] "RemoveContainer" containerID="ea2c8bb6092a65469cd9f4f4f69fe129da3eb452bda67a072956de35785b8d2b"
Sep 16 19:12:08.554085 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.554064    2578 scope.go:117] "RemoveContainer" containerID="6ff25e74ee38ed95b3932ca204c3e1264cdbdbb40f85c8572801c43bbf5f54a6"
Sep 16 19:12:08.563856 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.563825    2578 scope.go:117] "RemoveContainer" containerID="ea2c8bb6092a65469cd9f4f4f69fe129da3eb452bda67a072956de35785b8d2b"
Sep 16 19:12:08.564223 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:12:08.564201    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea2c8bb6092a65469cd9f4f4f69fe129da3eb452bda67a072956de35785b8d2b\": container with ID starting with ea2c8bb6092a65469cd9f4f4f69fe129da3eb452bda67a072956de35785b8d2b not found: ID does not exist" containerID="ea2c8bb6092a65469cd9f4f4f69fe129da3eb452bda67a072956de35785b8d2b"
Sep 16 19:12:08.564297 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.564233    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea2c8bb6092a65469cd9f4f4f69fe129da3eb452bda67a072956de35785b8d2b"} err="failed to get container status \"ea2c8bb6092a65469cd9f4f4f69fe129da3eb452bda67a072956de35785b8d2b\": rpc error: code = NotFound desc = could not find container \"ea2c8bb6092a65469cd9f4f4f69fe129da3eb452bda67a072956de35785b8d2b\": container with ID starting with ea2c8bb6092a65469cd9f4f4f69fe129da3eb452bda67a072956de35785b8d2b not found: ID does not exist"
Sep 16 19:12:08.564297 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.564253    2578 scope.go:117] "RemoveContainer" containerID="6ff25e74ee38ed95b3932ca204c3e1264cdbdbb40f85c8572801c43bbf5f54a6"
Sep 16 19:12:08.564533 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:12:08.564514    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ff25e74ee38ed95b3932ca204c3e1264cdbdbb40f85c8572801c43bbf5f54a6\": container with ID starting with 6ff25e74ee38ed95b3932ca204c3e1264cdbdbb40f85c8572801c43bbf5f54a6 not found: ID does not exist" containerID="6ff25e74ee38ed95b3932ca204c3e1264cdbdbb40f85c8572801c43bbf5f54a6"
Sep 16 19:12:08.564595 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.564543    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ff25e74ee38ed95b3932ca204c3e1264cdbdbb40f85c8572801c43bbf5f54a6"} err="failed to get container status \"6ff25e74ee38ed95b3932ca204c3e1264cdbdbb40f85c8572801c43bbf5f54a6\": rpc error: code = NotFound desc = could not find container \"6ff25e74ee38ed95b3932ca204c3e1264cdbdbb40f85c8572801c43bbf5f54a6\": container with ID starting with 6ff25e74ee38ed95b3932ca204c3e1264cdbdbb40f85c8572801c43bbf5f54a6 not found: ID does not exist"
Sep 16 19:12:08.570923 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.570897    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"]
Sep 16 19:12:08.576230 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.576206    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-c20219-openshift-default-6fdbbccd44-gkww7"]
Sep 16 19:12:08.709648 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:08.709614    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b163981d-51c3-4f96-9597-228867b865e5" path="/var/lib/kubelet/pods/b163981d-51c3-4f96-9597-228867b865e5/volumes"
Sep 16 19:12:21.591033 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:21.590996    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"]
Sep 16 19:12:21.591462 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:21.591251    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2" podUID="b46a4a50-fafe-4187-b140-220649dd54c0" containerName="istio-proxy" containerID="cri-o://79ca899248d49e8e0f57a702724626b2c0a10ea052a6c2e1c8a697242ad14ab8" gracePeriod=30
Sep 16 19:12:25.793983 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:25.793904    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 19:12:25.793983 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:25.793965    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2" podUID="b46a4a50-fafe-4187-b140-220649dd54c0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:12:26.847278 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.847245    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:12:26.975847 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.975813    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-workload-socket\") pod \"b46a4a50-fafe-4187-b140-220649dd54c0\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") "
Sep 16 19:12:26.976053 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.975863    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-istio-envoy\") pod \"b46a4a50-fafe-4187-b140-220649dd54c0\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") "
Sep 16 19:12:26.976053 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.975906    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b46a4a50-fafe-4187-b140-220649dd54c0-istio-podinfo\") pod \"b46a4a50-fafe-4187-b140-220649dd54c0\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") "
Sep 16 19:12:26.976053 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.975935    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b46a4a50-fafe-4187-b140-220649dd54c0-istio-token\") pod \"b46a4a50-fafe-4187-b140-220649dd54c0\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") "
Sep 16 19:12:26.976053 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.975998    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-credential-socket\") pod \"b46a4a50-fafe-4187-b140-220649dd54c0\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") "
Sep 16 19:12:26.976053 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.976041    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l76h2\" (UniqueName: \"kubernetes.io/projected/b46a4a50-fafe-4187-b140-220649dd54c0-kube-api-access-l76h2\") pod \"b46a4a50-fafe-4187-b140-220649dd54c0\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") "
Sep 16 19:12:26.976368 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.976059    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-workload-certs\") pod \"b46a4a50-fafe-4187-b140-220649dd54c0\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") "
Sep 16 19:12:26.976368 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.976077    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b46a4a50-fafe-4187-b140-220649dd54c0-istiod-ca-cert\") pod \"b46a4a50-fafe-4187-b140-220649dd54c0\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") "
Sep 16 19:12:26.976368 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.976112    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-istio-data\") pod \"b46a4a50-fafe-4187-b140-220649dd54c0\" (UID: \"b46a4a50-fafe-4187-b140-220649dd54c0\") "
Sep 16 19:12:26.976368 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.976134    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b46a4a50-fafe-4187-b140-220649dd54c0" (UID: "b46a4a50-fafe-4187-b140-220649dd54c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:12:26.976569 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.976413    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b46a4a50-fafe-4187-b140-220649dd54c0" (UID: "b46a4a50-fafe-4187-b140-220649dd54c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:12:26.976569 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.976438    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b46a4a50-fafe-4187-b140-220649dd54c0" (UID: "b46a4a50-fafe-4187-b140-220649dd54c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:12:26.976569 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.976551    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b46a4a50-fafe-4187-b140-220649dd54c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b46a4a50-fafe-4187-b140-220649dd54c0" (UID: "b46a4a50-fafe-4187-b140-220649dd54c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:12:26.976726 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.976578    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "b46a4a50-fafe-4187-b140-220649dd54c0" (UID: "b46a4a50-fafe-4187-b140-220649dd54c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:12:26.976726 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.976614    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:26.976726 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.976633    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:26.976726 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.976648    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:26.978301 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.978271    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b46a4a50-fafe-4187-b140-220649dd54c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "b46a4a50-fafe-4187-b140-220649dd54c0" (UID: "b46a4a50-fafe-4187-b140-220649dd54c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:12:26.978449 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.978419    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b46a4a50-fafe-4187-b140-220649dd54c0" (UID: "b46a4a50-fafe-4187-b140-220649dd54c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:12:26.978686 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.978659    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b46a4a50-fafe-4187-b140-220649dd54c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b46a4a50-fafe-4187-b140-220649dd54c0" (UID: "b46a4a50-fafe-4187-b140-220649dd54c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:12:26.978686 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:26.978672    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b46a4a50-fafe-4187-b140-220649dd54c0-kube-api-access-l76h2" (OuterVolumeSpecName: "kube-api-access-l76h2") pod "b46a4a50-fafe-4187-b140-220649dd54c0" (UID: "b46a4a50-fafe-4187-b140-220649dd54c0"). InnerVolumeSpecName "kube-api-access-l76h2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:12:27.077646 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.077610    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:27.077646 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.077639    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b46a4a50-fafe-4187-b140-220649dd54c0-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:27.077646 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.077648    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b46a4a50-fafe-4187-b140-220649dd54c0-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:27.077873 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.077658    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l76h2\" (UniqueName: \"kubernetes.io/projected/b46a4a50-fafe-4187-b140-220649dd54c0-kube-api-access-l76h2\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:27.077873 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.077668    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b46a4a50-fafe-4187-b140-220649dd54c0-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:27.077873 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.077676    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b46a4a50-fafe-4187-b140-220649dd54c0-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:27.675415 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.675382    2578 generic.go:356] "Generic (PLEG): container finished" podID="b46a4a50-fafe-4187-b140-220649dd54c0" containerID="79ca899248d49e8e0f57a702724626b2c0a10ea052a6c2e1c8a697242ad14ab8" exitCode=0
Sep 16 19:12:27.675616 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.675452    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"
Sep 16 19:12:27.675616 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.675466    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2" event={"ID":"b46a4a50-fafe-4187-b140-220649dd54c0","Type":"ContainerDied","Data":"79ca899248d49e8e0f57a702724626b2c0a10ea052a6c2e1c8a697242ad14ab8"}
Sep 16 19:12:27.675616 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.675565    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2" event={"ID":"b46a4a50-fafe-4187-b140-220649dd54c0","Type":"ContainerDied","Data":"bdccff54a5c6f7f44c0bf468c75eeb52e9ad41ab917a224343fa1a69b731ef36"}
Sep 16 19:12:27.675616 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.675596    2578 scope.go:117] "RemoveContainer" containerID="79ca899248d49e8e0f57a702724626b2c0a10ea052a6c2e1c8a697242ad14ab8"
Sep 16 19:12:27.686091 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.686074    2578 scope.go:117] "RemoveContainer" containerID="a7fa671fbdee4c2190817b6175e3acec8828d61b33886492d38b406d595b3fbb"
Sep 16 19:12:27.694913 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.694894    2578 scope.go:117] "RemoveContainer" containerID="79ca899248d49e8e0f57a702724626b2c0a10ea052a6c2e1c8a697242ad14ab8"
Sep 16 19:12:27.695165 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:12:27.695146    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79ca899248d49e8e0f57a702724626b2c0a10ea052a6c2e1c8a697242ad14ab8\": container with ID starting with 79ca899248d49e8e0f57a702724626b2c0a10ea052a6c2e1c8a697242ad14ab8 not found: ID does not exist" containerID="79ca899248d49e8e0f57a702724626b2c0a10ea052a6c2e1c8a697242ad14ab8"
Sep 16 19:12:27.695257 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.695189    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79ca899248d49e8e0f57a702724626b2c0a10ea052a6c2e1c8a697242ad14ab8"} err="failed to get container status \"79ca899248d49e8e0f57a702724626b2c0a10ea052a6c2e1c8a697242ad14ab8\": rpc error: code = NotFound desc = could not find container \"79ca899248d49e8e0f57a702724626b2c0a10ea052a6c2e1c8a697242ad14ab8\": container with ID starting with 79ca899248d49e8e0f57a702724626b2c0a10ea052a6c2e1c8a697242ad14ab8 not found: ID does not exist"
Sep 16 19:12:27.695257 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.695207    2578 scope.go:117] "RemoveContainer" containerID="a7fa671fbdee4c2190817b6175e3acec8828d61b33886492d38b406d595b3fbb"
Sep 16 19:12:27.695488 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:12:27.695467    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7fa671fbdee4c2190817b6175e3acec8828d61b33886492d38b406d595b3fbb\": container with ID starting with a7fa671fbdee4c2190817b6175e3acec8828d61b33886492d38b406d595b3fbb not found: ID does not exist" containerID="a7fa671fbdee4c2190817b6175e3acec8828d61b33886492d38b406d595b3fbb"
Sep 16 19:12:27.695526 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.695494    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7fa671fbdee4c2190817b6175e3acec8828d61b33886492d38b406d595b3fbb"} err="failed to get container status \"a7fa671fbdee4c2190817b6175e3acec8828d61b33886492d38b406d595b3fbb\": rpc error: code = NotFound desc = could not find container \"a7fa671fbdee4c2190817b6175e3acec8828d61b33886492d38b406d595b3fbb\": container with ID starting with a7fa671fbdee4c2190817b6175e3acec8828d61b33886492d38b406d595b3fbb not found: ID does not exist"
Sep 16 19:12:27.701441 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.701418    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"]
Sep 16 19:12:27.704858 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:27.704837    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-58bf51-openshift-default-8689b45d6-dpcj2"]
Sep 16 19:12:28.709373 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:28.709324    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b46a4a50-fafe-4187-b140-220649dd54c0" path="/var/lib/kubelet/pods/b46a4a50-fafe-4187-b140-220649dd54c0/volumes"
Sep 16 19:12:50.711382 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:50.711349    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"]
Sep 16 19:12:51.934305 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:51.934272    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"]
Sep 16 19:12:54.146572 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:54.146532    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"]
Sep 16 19:12:54.147091 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:54.146785    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t" podUID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" containerName="istio-proxy" containerID="cri-o://0a9c3177f65844b4805b93190facdd9af4f701878a18cf4cf06a28404179bef6" gracePeriod=30
Sep 16 19:12:59.405090 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.405064    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 19:12:59.509150 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.509053    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-workload-certs\") pod \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") "
Sep 16 19:12:59.509150 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.509129    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istiod-ca-cert\") pod \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") "
Sep 16 19:12:59.509446 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.509208    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-workload-socket\") pod \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") "
Sep 16 19:12:59.509446 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.509307    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxvpv\" (UniqueName: \"kubernetes.io/projected/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-kube-api-access-jxvpv\") pod \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") "
Sep 16 19:12:59.509446 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.509365    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-token\") pod \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") "
Sep 16 19:12:59.509446 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.509409    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-data\") pod \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") "
Sep 16 19:12:59.509446 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.509427    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" (UID: "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:12:59.509703 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.509464    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-envoy\") pod \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") "
Sep 16 19:12:59.509703 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.509510    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" (UID: "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:12:59.509703 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.509552    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-podinfo\") pod \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") "
Sep 16 19:12:59.509703 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.509582    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-credential-socket\") pod \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\" (UID: \"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a\") "
Sep 16 19:12:59.509900 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.509717    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-data" (OuterVolumeSpecName: "istio-data") pod "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" (UID: "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:12:59.509958 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.509897    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" (UID: "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:12:59.510040 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.510014    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:59.510040 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.510032    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:59.510129 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.510041    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:59.510129 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.510031    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" (UID: "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:12:59.510129 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.510052    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:59.511805 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.511776    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" (UID: "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:12:59.511942 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.511920    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-token" (OuterVolumeSpecName: "istio-token") pod "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" (UID: "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:12:59.512064 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.512044    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" (UID: "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:12:59.512121 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.512048    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-kube-api-access-jxvpv" (OuterVolumeSpecName: "kube-api-access-jxvpv") pod "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" (UID: "da1b180f-d138-4a97-9fe4-1bc43cfeaa4a"). InnerVolumeSpecName "kube-api-access-jxvpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:12:59.611532 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.611495    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:59.611532 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.611525    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:59.611532 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.611535    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:59.611786 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.611544    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxvpv\" (UniqueName: \"kubernetes.io/projected/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-kube-api-access-jxvpv\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:59.611786 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.611553    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:12:59.880683 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.880649    2578 generic.go:356] "Generic (PLEG): container finished" podID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" containerID="0a9c3177f65844b4805b93190facdd9af4f701878a18cf4cf06a28404179bef6" exitCode=0
Sep 16 19:12:59.880895 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.880700    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"
Sep 16 19:12:59.880895 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.880738    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t" event={"ID":"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a","Type":"ContainerDied","Data":"0a9c3177f65844b4805b93190facdd9af4f701878a18cf4cf06a28404179bef6"}
Sep 16 19:12:59.880895 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.880804    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t" event={"ID":"da1b180f-d138-4a97-9fe4-1bc43cfeaa4a","Type":"ContainerDied","Data":"33f9f3508a46cab900eef6b6cac63ddbf44303ee50bb1caefb60d3c73f6b003f"}
Sep 16 19:12:59.880895 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.880837    2578 scope.go:117] "RemoveContainer" containerID="0a9c3177f65844b4805b93190facdd9af4f701878a18cf4cf06a28404179bef6"
Sep 16 19:12:59.890592 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.890576    2578 scope.go:117] "RemoveContainer" containerID="bbc2d1ef731853d906413f8a803096c77fed7a9ef916450b95e3db4830b04569"
Sep 16 19:12:59.899370 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.899351    2578 scope.go:117] "RemoveContainer" containerID="0a9c3177f65844b4805b93190facdd9af4f701878a18cf4cf06a28404179bef6"
Sep 16 19:12:59.899640 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:12:59.899623    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a9c3177f65844b4805b93190facdd9af4f701878a18cf4cf06a28404179bef6\": container with ID starting with 0a9c3177f65844b4805b93190facdd9af4f701878a18cf4cf06a28404179bef6 not found: ID does not exist" containerID="0a9c3177f65844b4805b93190facdd9af4f701878a18cf4cf06a28404179bef6"
Sep 16 19:12:59.899709 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.899647    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a9c3177f65844b4805b93190facdd9af4f701878a18cf4cf06a28404179bef6"} err="failed to get container status \"0a9c3177f65844b4805b93190facdd9af4f701878a18cf4cf06a28404179bef6\": rpc error: code = NotFound desc = could not find container \"0a9c3177f65844b4805b93190facdd9af4f701878a18cf4cf06a28404179bef6\": container with ID starting with 0a9c3177f65844b4805b93190facdd9af4f701878a18cf4cf06a28404179bef6 not found: ID does not exist"
Sep 16 19:12:59.899709 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.899663    2578 scope.go:117] "RemoveContainer" containerID="bbc2d1ef731853d906413f8a803096c77fed7a9ef916450b95e3db4830b04569"
Sep 16 19:12:59.899911 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:12:59.899894    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbc2d1ef731853d906413f8a803096c77fed7a9ef916450b95e3db4830b04569\": container with ID starting with bbc2d1ef731853d906413f8a803096c77fed7a9ef916450b95e3db4830b04569 not found: ID does not exist" containerID="bbc2d1ef731853d906413f8a803096c77fed7a9ef916450b95e3db4830b04569"
Sep 16 19:12:59.899951 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.899917    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc2d1ef731853d906413f8a803096c77fed7a9ef916450b95e3db4830b04569"} err="failed to get container status \"bbc2d1ef731853d906413f8a803096c77fed7a9ef916450b95e3db4830b04569\": rpc error: code = NotFound desc = could not find container \"bbc2d1ef731853d906413f8a803096c77fed7a9ef916450b95e3db4830b04569\": container with ID starting with bbc2d1ef731853d906413f8a803096c77fed7a9ef916450b95e3db4830b04569 not found: ID does not exist"
Sep 16 19:12:59.907419 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.907397    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"]
Sep 16 19:12:59.911434 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:12:59.911414    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-c090c8-openshift-default-6c574bd5b6-k5n8t"]
Sep 16 19:13:00.709534 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:00.709501    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" path="/var/lib/kubelet/pods/da1b180f-d138-4a97-9fe4-1bc43cfeaa4a/volumes"
Sep 16 19:13:05.730416 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:05.730369    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm" podUID="c2e983d4-f3c1-495e-af59-d149ef1b57c1" containerName="main" containerID="cri-o://a5b89e5e8e48e01b82d1758356b4e4a905d2bcf007d8bb3213beb84b3a0e48c4" gracePeriod=45
Sep 16 19:13:05.923277 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:05.923248    2578 generic.go:356] "Generic (PLEG): container finished" podID="c2e983d4-f3c1-495e-af59-d149ef1b57c1" containerID="a5b89e5e8e48e01b82d1758356b4e4a905d2bcf007d8bb3213beb84b3a0e48c4" exitCode=0
Sep 16 19:13:05.923434 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:05.923309    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm" event={"ID":"c2e983d4-f3c1-495e-af59-d149ef1b57c1","Type":"ContainerDied","Data":"a5b89e5e8e48e01b82d1758356b4e4a905d2bcf007d8bb3213beb84b3a0e48c4"}
Sep 16 19:13:05.983868 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:05.983807    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:13:06.083860 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.083831    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-dshm\") pod \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") "
Sep 16 19:13:06.083860 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.083870    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-home\") pod \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") "
Sep 16 19:13:06.084124 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.083923    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jpm8\" (UniqueName: \"kubernetes.io/projected/c2e983d4-f3c1-495e-af59-d149ef1b57c1-kube-api-access-7jpm8\") pod \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") "
Sep 16 19:13:06.084124 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.083947    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2e983d4-f3c1-495e-af59-d149ef1b57c1-tls-certs\") pod \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") "
Sep 16 19:13:06.084124 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.083985    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-tmp-dir\") pod \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") "
Sep 16 19:13:06.084124 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.084025    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-model-cache\") pod \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\" (UID: \"c2e983d4-f3c1-495e-af59-d149ef1b57c1\") "
Sep 16 19:13:06.084371 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.084263    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-home" (OuterVolumeSpecName: "home") pod "c2e983d4-f3c1-495e-af59-d149ef1b57c1" (UID: "c2e983d4-f3c1-495e-af59-d149ef1b57c1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:13:06.084437 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.084378    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c2e983d4-f3c1-495e-af59-d149ef1b57c1" (UID: "c2e983d4-f3c1-495e-af59-d149ef1b57c1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:13:06.084509 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.084487    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-model-cache" (OuterVolumeSpecName: "model-cache") pod "c2e983d4-f3c1-495e-af59-d149ef1b57c1" (UID: "c2e983d4-f3c1-495e-af59-d149ef1b57c1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:13:06.086262 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.086238    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-dshm" (OuterVolumeSpecName: "dshm") pod "c2e983d4-f3c1-495e-af59-d149ef1b57c1" (UID: "c2e983d4-f3c1-495e-af59-d149ef1b57c1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:13:06.086544 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.086508    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2e983d4-f3c1-495e-af59-d149ef1b57c1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c2e983d4-f3c1-495e-af59-d149ef1b57c1" (UID: "c2e983d4-f3c1-495e-af59-d149ef1b57c1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:13:06.086544 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.086512    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2e983d4-f3c1-495e-af59-d149ef1b57c1-kube-api-access-7jpm8" (OuterVolumeSpecName: "kube-api-access-7jpm8") pod "c2e983d4-f3c1-495e-af59-d149ef1b57c1" (UID: "c2e983d4-f3c1-495e-af59-d149ef1b57c1"). InnerVolumeSpecName "kube-api-access-7jpm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:13:06.184858 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.184821    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-model-cache\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:13:06.184858 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.184853    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-dshm\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:13:06.184858 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.184861    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-home\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:13:06.185096 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.184870    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jpm8\" (UniqueName: \"kubernetes.io/projected/c2e983d4-f3c1-495e-af59-d149ef1b57c1-kube-api-access-7jpm8\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:13:06.185096 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.184879    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2e983d4-f3c1-495e-af59-d149ef1b57c1-tls-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:13:06.185096 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.184887    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2e983d4-f3c1-495e-af59-d149ef1b57c1-tmp-dir\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:13:06.931347 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.931311    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm" event={"ID":"c2e983d4-f3c1-495e-af59-d149ef1b57c1","Type":"ContainerDied","Data":"ceb3055061c9f74aec6762cc491694d15229441a08642885ca7da1246caf4eb8"}
Sep 16 19:13:06.931728 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.931361    2578 scope.go:117] "RemoveContainer" containerID="a5b89e5e8e48e01b82d1758356b4e4a905d2bcf007d8bb3213beb84b3a0e48c4"
Sep 16 19:13:06.931728 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.931328    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"
Sep 16 19:13:06.950421 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.950396    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"]
Sep 16 19:13:06.953041 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.953014    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ca3f9a/auth-test-model-ca3f9a-kserve-8cbcd6d8c-fhkxm"]
Sep 16 19:13:06.953142 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:06.953074    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v" podUID="44a6241f-65d7-465d-9300-5237a7c871b3" containerName="main" containerID="cri-o://f53122563a9b5c5e0229869ac2cebb65901a88cb78e6b76c0c5eca8287d2c8a3" gracePeriod=45
Sep 16 19:13:07.202238 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.202208    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:13:07.295831 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.295723    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-tmp-dir\") pod \"44a6241f-65d7-465d-9300-5237a7c871b3\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") "
Sep 16 19:13:07.295831 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.295775    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-dshm\") pod \"44a6241f-65d7-465d-9300-5237a7c871b3\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") "
Sep 16 19:13:07.295831 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.295813    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-home\") pod \"44a6241f-65d7-465d-9300-5237a7c871b3\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") "
Sep 16 19:13:07.296456 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.295872    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44a6241f-65d7-465d-9300-5237a7c871b3-tls-certs\") pod \"44a6241f-65d7-465d-9300-5237a7c871b3\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") "
Sep 16 19:13:07.296456 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.295914    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvx4t\" (UniqueName: \"kubernetes.io/projected/44a6241f-65d7-465d-9300-5237a7c871b3-kube-api-access-qvx4t\") pod \"44a6241f-65d7-465d-9300-5237a7c871b3\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") "
Sep 16 19:13:07.296456 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.296016    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-model-cache\") pod \"44a6241f-65d7-465d-9300-5237a7c871b3\" (UID: \"44a6241f-65d7-465d-9300-5237a7c871b3\") "
Sep 16 19:13:07.296456 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.296095    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "44a6241f-65d7-465d-9300-5237a7c871b3" (UID: "44a6241f-65d7-465d-9300-5237a7c871b3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:13:07.296456 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.296395    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-tmp-dir\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:13:07.296730 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.296589    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-home" (OuterVolumeSpecName: "home") pod "44a6241f-65d7-465d-9300-5237a7c871b3" (UID: "44a6241f-65d7-465d-9300-5237a7c871b3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:13:07.296730 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.296627    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-model-cache" (OuterVolumeSpecName: "model-cache") pod "44a6241f-65d7-465d-9300-5237a7c871b3" (UID: "44a6241f-65d7-465d-9300-5237a7c871b3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:13:07.299431 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.299404    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-dshm" (OuterVolumeSpecName: "dshm") pod "44a6241f-65d7-465d-9300-5237a7c871b3" (UID: "44a6241f-65d7-465d-9300-5237a7c871b3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:13:07.299540 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.299492    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44a6241f-65d7-465d-9300-5237a7c871b3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "44a6241f-65d7-465d-9300-5237a7c871b3" (UID: "44a6241f-65d7-465d-9300-5237a7c871b3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:13:07.299634 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.299613    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44a6241f-65d7-465d-9300-5237a7c871b3-kube-api-access-qvx4t" (OuterVolumeSpecName: "kube-api-access-qvx4t") pod "44a6241f-65d7-465d-9300-5237a7c871b3" (UID: "44a6241f-65d7-465d-9300-5237a7c871b3"). InnerVolumeSpecName "kube-api-access-qvx4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:13:07.397698 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.397657    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-model-cache\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:13:07.397698 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.397692    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-dshm\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:13:07.397698 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.397700    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44a6241f-65d7-465d-9300-5237a7c871b3-home\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:13:07.397698 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.397707    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44a6241f-65d7-465d-9300-5237a7c871b3-tls-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:13:07.397969 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.397716    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvx4t\" (UniqueName: \"kubernetes.io/projected/44a6241f-65d7-465d-9300-5237a7c871b3-kube-api-access-qvx4t\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:13:07.938717 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.938685    2578 generic.go:356] "Generic (PLEG): container finished" podID="44a6241f-65d7-465d-9300-5237a7c871b3" containerID="f53122563a9b5c5e0229869ac2cebb65901a88cb78e6b76c0c5eca8287d2c8a3" exitCode=0
Sep 16 19:13:07.939117 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.938751    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"
Sep 16 19:13:07.939117 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.938771    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v" event={"ID":"44a6241f-65d7-465d-9300-5237a7c871b3","Type":"ContainerDied","Data":"f53122563a9b5c5e0229869ac2cebb65901a88cb78e6b76c0c5eca8287d2c8a3"}
Sep 16 19:13:07.939117 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.938815    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v" event={"ID":"44a6241f-65d7-465d-9300-5237a7c871b3","Type":"ContainerDied","Data":"075c93f3ddf3996eb0f3032288dda7c92e7373f6f9756aeb6a9e6d10f23f457b"}
Sep 16 19:13:07.939117 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.938832    2578 scope.go:117] "RemoveContainer" containerID="f53122563a9b5c5e0229869ac2cebb65901a88cb78e6b76c0c5eca8287d2c8a3"
Sep 16 19:13:07.948516 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.948493    2578 scope.go:117] "RemoveContainer" containerID="f53122563a9b5c5e0229869ac2cebb65901a88cb78e6b76c0c5eca8287d2c8a3"
Sep 16 19:13:07.948839 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:13:07.948820    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f53122563a9b5c5e0229869ac2cebb65901a88cb78e6b76c0c5eca8287d2c8a3\": container with ID starting with f53122563a9b5c5e0229869ac2cebb65901a88cb78e6b76c0c5eca8287d2c8a3 not found: ID does not exist" containerID="f53122563a9b5c5e0229869ac2cebb65901a88cb78e6b76c0c5eca8287d2c8a3"
Sep 16 19:13:07.948909 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.948846    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f53122563a9b5c5e0229869ac2cebb65901a88cb78e6b76c0c5eca8287d2c8a3"} err="failed to get container status \"f53122563a9b5c5e0229869ac2cebb65901a88cb78e6b76c0c5eca8287d2c8a3\": rpc error: code = NotFound desc = could not find container \"f53122563a9b5c5e0229869ac2cebb65901a88cb78e6b76c0c5eca8287d2c8a3\": container with ID starting with f53122563a9b5c5e0229869ac2cebb65901a88cb78e6b76c0c5eca8287d2c8a3 not found: ID does not exist"
Sep 16 19:13:07.965798 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.965764    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"]
Sep 16 19:13:07.969954 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:07.969929    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-2dc9fc/auth-test-model-2dc9fc-kserve-6b6d8648b7-rk78v"]
Sep 16 19:13:08.712040 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:08.712007    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44a6241f-65d7-465d-9300-5237a7c871b3" path="/var/lib/kubelet/pods/44a6241f-65d7-465d-9300-5237a7c871b3/volumes"
Sep 16 19:13:08.712468 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:13:08.712452    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2e983d4-f3c1-495e-af59-d149ef1b57c1" path="/var/lib/kubelet/pods/c2e983d4-f3c1-495e-af59-d149ef1b57c1/volumes"
Sep 16 19:14:32.874913 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:14:32.874883    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 19:14:32.878469 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:14:32.878446    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 19:14:32.881012 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:14:32.880992    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 19:14:32.884319 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:14:32.884300    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 19:14:33.227056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:14:33.226967    2578 scope.go:117] "RemoveContainer" containerID="33b433eb437803b68971773a4bcd1c80b9bad976b61a276bc31a767a53b1ab0e"
Sep 16 19:15:00.191742 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.191708    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826435-gfkxv"]
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192108    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b163981d-51c3-4f96-9597-228867b865e5" containerName="istio-proxy"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192119    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b163981d-51c3-4f96-9597-228867b865e5" containerName="istio-proxy"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192133    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" containerName="istio-proxy"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192139    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" containerName="istio-proxy"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192145    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" containerName="istio-proxy"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192151    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" containerName="istio-proxy"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192160    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2e983d4-f3c1-495e-af59-d149ef1b57c1" containerName="main"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192166    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2e983d4-f3c1-495e-af59-d149ef1b57c1" containerName="main"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192190    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44a6241f-65d7-465d-9300-5237a7c871b3" containerName="main"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192198    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="44a6241f-65d7-465d-9300-5237a7c871b3" containerName="main"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192206    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" containerName="istio-proxy"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192211    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" containerName="istio-proxy"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192220    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9855a54a-d7e8-42b1-aed2-a2ed874b152a" containerName="main"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192225    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9855a54a-d7e8-42b1-aed2-a2ed874b152a" containerName="main"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192232    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c292ec82-49cf-4c8d-9072-6eff2febbe63" containerName="istio-proxy"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192238    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c292ec82-49cf-4c8d-9072-6eff2febbe63" containerName="istio-proxy"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192245    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b46a4a50-fafe-4187-b140-220649dd54c0" containerName="istio-proxy"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192250    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b46a4a50-fafe-4187-b140-220649dd54c0" containerName="istio-proxy"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192257    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="376e752f-e054-4b59-8249-55e1112454e8" containerName="istio-proxy"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192262    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="376e752f-e054-4b59-8249-55e1112454e8" containerName="istio-proxy"
Sep 16 19:15:00.192259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192270    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="376e752f-e054-4b59-8249-55e1112454e8" containerName="istio-proxy"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192275    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="376e752f-e054-4b59-8249-55e1112454e8" containerName="istio-proxy"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192288    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b46a4a50-fafe-4187-b140-220649dd54c0" containerName="istio-proxy"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192295    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b46a4a50-fafe-4187-b140-220649dd54c0" containerName="istio-proxy"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192300    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c292ec82-49cf-4c8d-9072-6eff2febbe63" containerName="istio-proxy"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192305    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c292ec82-49cf-4c8d-9072-6eff2febbe63" containerName="istio-proxy"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192311    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" containerName="istio-proxy"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192316    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" containerName="istio-proxy"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192371    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c292ec82-49cf-4c8d-9072-6eff2febbe63" containerName="istio-proxy"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192379    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" containerName="istio-proxy"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192384    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="44a6241f-65d7-465d-9300-5237a7c871b3" containerName="main"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192388    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" containerName="istio-proxy"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192393    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9855a54a-d7e8-42b1-aed2-a2ed874b152a" containerName="main"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192398    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="376e752f-e054-4b59-8249-55e1112454e8" containerName="istio-proxy"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192403    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b46a4a50-fafe-4187-b140-220649dd54c0" containerName="istio-proxy"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192408    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b163981d-51c3-4f96-9597-228867b865e5" containerName="istio-proxy"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192416    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b46a4a50-fafe-4187-b140-220649dd54c0" containerName="istio-proxy"
Sep 16 19:15:00.192972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.192421    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2e983d4-f3c1-495e-af59-d149ef1b57c1" containerName="main"
Sep 16 19:15:00.195603 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.195583    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826435-gfkxv"
Sep 16 19:15:00.201644 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.201619    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nmr62\""
Sep 16 19:15:00.205153 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.205130    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826435-gfkxv"
Sep 16 19:15:00.216544 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.216519    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826435-gfkxv"]
Sep 16 19:15:00.482547 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.482519    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826435-gfkxv"]
Sep 16 19:15:00.484672 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:15:00.484645    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod976e845e_b2fc_436b_9c41_60b652bb5354.slice/crio-9fd386c35428f599251382c726cb16294cf82b3fbc10344df9de12c9a5906219 WatchSource:0}: Error finding container 9fd386c35428f599251382c726cb16294cf82b3fbc10344df9de12c9a5906219: Status 404 returned error can't find the container with id 9fd386c35428f599251382c726cb16294cf82b3fbc10344df9de12c9a5906219
Sep 16 19:15:00.635245 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:00.635210    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826435-gfkxv" event={"ID":"976e845e-b2fc-436b-9c41-60b652bb5354","Type":"ContainerStarted","Data":"9fd386c35428f599251382c726cb16294cf82b3fbc10344df9de12c9a5906219"}
Sep 16 19:15:01.650635 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:01.650598    2578 generic.go:356] "Generic (PLEG): container finished" podID="976e845e-b2fc-436b-9c41-60b652bb5354" containerID="09ad96a1846d4a9f901724a3f6c2786d25c5638b63bb55a718f60d10aa41219b" exitCode=0
Sep 16 19:15:01.651030 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:01.650641    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826435-gfkxv" event={"ID":"976e845e-b2fc-436b-9c41-60b652bb5354","Type":"ContainerDied","Data":"09ad96a1846d4a9f901724a3f6c2786d25c5638b63bb55a718f60d10aa41219b"}
Sep 16 19:15:02.784084 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:02.784058    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826435-gfkxv"
Sep 16 19:15:03.663925 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:03.663898    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826435-gfkxv"
Sep 16 19:15:03.663925 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:03.663919    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826435-gfkxv" event={"ID":"976e845e-b2fc-436b-9c41-60b652bb5354","Type":"ContainerDied","Data":"9fd386c35428f599251382c726cb16294cf82b3fbc10344df9de12c9a5906219"}
Sep 16 19:15:03.664147 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:03.663966    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9fd386c35428f599251382c726cb16294cf82b3fbc10344df9de12c9a5906219"
Sep 16 19:15:39.234068 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.234036    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"]
Sep 16 19:15:39.234559 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.234453    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b163981d-51c3-4f96-9597-228867b865e5" containerName="istio-proxy"
Sep 16 19:15:39.234559 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.234465    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b163981d-51c3-4f96-9597-228867b865e5" containerName="istio-proxy"
Sep 16 19:15:39.234559 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.234494    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="976e845e-b2fc-436b-9c41-60b652bb5354" containerName="cleanup"
Sep 16 19:15:39.234559 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.234499    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="976e845e-b2fc-436b-9c41-60b652bb5354" containerName="cleanup"
Sep 16 19:15:39.234699 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.234565    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="976e845e-b2fc-436b-9c41-60b652bb5354" containerName="cleanup"
Sep 16 19:15:39.234699 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.234573    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c292ec82-49cf-4c8d-9072-6eff2febbe63" containerName="istio-proxy"
Sep 16 19:15:39.234699 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.234580    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="da1b180f-d138-4a97-9fe4-1bc43cfeaa4a" containerName="istio-proxy"
Sep 16 19:15:39.234699 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.234587    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3a3aca2-0d4e-4b40-8b54-f4ca27b3b9a6" containerName="istio-proxy"
Sep 16 19:15:39.234699 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.234592    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="376e752f-e054-4b59-8249-55e1112454e8" containerName="istio-proxy"
Sep 16 19:15:39.234699 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.234599    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b163981d-51c3-4f96-9597-228867b865e5" containerName="istio-proxy"
Sep 16 19:15:39.237743 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.237724    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.241930 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.241894    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-4992ef-openshift-default-dockercfg-npw4c\""
Sep 16 19:15:39.278162 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.278097    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"]
Sep 16 19:15:39.283676 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.283649    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-token\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.283836 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.283714    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-credential-socket\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.283836 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.283765    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-envoy\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.283836 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.283814    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac10eb3a-a871-4a69-aecd-02c2185b56be-istiod-ca-cert\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.284021 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.283852    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-podinfo\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.284021 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.283872    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-data\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.284021 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.283887    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzddv\" (UniqueName: \"kubernetes.io/projected/ac10eb3a-a871-4a69-aecd-02c2185b56be-kube-api-access-fzddv\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.284021 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.283907    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-workload-certs\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.284021 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.283969    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-workload-socket\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.384565 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.384523    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-credential-socket\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.384776 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.384580    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-envoy\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.384776 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.384651    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac10eb3a-a871-4a69-aecd-02c2185b56be-istiod-ca-cert\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.384776 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.384690    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-podinfo\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.384776 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.384707    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-data\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.384776 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.384723    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzddv\" (UniqueName: \"kubernetes.io/projected/ac10eb3a-a871-4a69-aecd-02c2185b56be-kube-api-access-fzddv\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.384776 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.384744    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-workload-certs\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.384776 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.384772    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-workload-socket\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.385156 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.385108    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-token\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.385156 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.385125    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-workload-socket\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.385316 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.385263    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-data\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.385380 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.385311    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-workload-certs\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.385543 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.385518    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac10eb3a-a871-4a69-aecd-02c2185b56be-istiod-ca-cert\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.385543 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.385528    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-credential-socket\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.387422 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.387397    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-envoy\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.387488 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.387473    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-podinfo\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.420299 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.420268    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-token\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.422230 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.422209    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzddv\" (UniqueName: \"kubernetes.io/projected/ac10eb3a-a871-4a69-aecd-02c2185b56be-kube-api-access-fzddv\") pod \"e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") " pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.549629 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.549548    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:39.832895 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.832870    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"]
Sep 16 19:15:39.834421 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:15:39.834391    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac10eb3a_a871_4a69_aecd_02c2185b56be.slice/crio-99be4a91ce9c935165468166b2cd9f8dedf2420c924bac214bf81e5339c9cfc4 WatchSource:0}: Error finding container 99be4a91ce9c935165468166b2cd9f8dedf2420c924bac214bf81e5339c9cfc4: Status 404 returned error can't find the container with id 99be4a91ce9c935165468166b2cd9f8dedf2420c924bac214bf81e5339c9cfc4
Sep 16 19:15:39.836747 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.836713    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:15:39.836832 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.836778    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:15:39.836832 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.836805    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:15:39.884988 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:39.884960    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9" event={"ID":"ac10eb3a-a871-4a69-aecd-02c2185b56be","Type":"ContainerStarted","Data":"99be4a91ce9c935165468166b2cd9f8dedf2420c924bac214bf81e5339c9cfc4"}
Sep 16 19:15:40.892770 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:40.892733    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9" event={"ID":"ac10eb3a-a871-4a69-aecd-02c2185b56be","Type":"ContainerStarted","Data":"335934a5879d66efe9d0ae862a6a51c37c82936c1173aa23c122284104d11bf5"}
Sep 16 19:15:40.916397 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:40.916342    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9" podStartSLOduration=1.916324394 podStartE2EDuration="1.916324394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:15:40.914981252 +0000 UTC m=+2168.702545428" watchObservedRunningTime="2026-09-16 19:15:40.916324394 +0000 UTC m=+2168.703888549"
Sep 16 19:15:41.550444 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:41.550407    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:41.555576 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:41.555549    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:41.898794 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:41.898755    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:41.899698 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:41.899679    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:15:57.940412 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:57.940367    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 16 19:15:57.941788 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:57.940447    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9" podUID="ac10eb3a-a871-4a69-aecd-02c2185b56be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 16 19:15:59.018937 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:59.018900    2578 generic.go:356] "Generic (PLEG): container finished" podID="ac10eb3a-a871-4a69-aecd-02c2185b56be" containerID="335934a5879d66efe9d0ae862a6a51c37c82936c1173aa23c122284104d11bf5" exitCode=0
Sep 16 19:15:59.019365 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:59.018971    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9" event={"ID":"ac10eb3a-a871-4a69-aecd-02c2185b56be","Type":"ContainerDied","Data":"335934a5879d66efe9d0ae862a6a51c37c82936c1173aa23c122284104d11bf5"}
Sep 16 19:15:59.019430 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:59.019417    2578 scope.go:117] "RemoveContainer" containerID="335934a5879d66efe9d0ae862a6a51c37c82936c1173aa23c122284104d11bf5"
Sep 16 19:15:59.020556 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:59.020523    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:15:59.020672 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:59.020600    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:15:59.020672 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:15:59.020637    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:16:00.027367 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:00.027330    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9" event={"ID":"ac10eb3a-a871-4a69-aecd-02c2185b56be","Type":"ContainerStarted","Data":"2e380695909c069d3c144ae884ec6e216aafd7109cc53495b16a4eeacc8156a4"}
Sep 16 19:16:00.550470 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:00.550376    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:16:00.550777 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:00.550753    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Sep 16 19:16:00.550918 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:00.550894    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9" podUID="ac10eb3a-a871-4a69-aecd-02c2185b56be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Sep 16 19:16:01.551252 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:01.550729    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Sep 16 19:16:01.551252 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:01.550814    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9" podUID="ac10eb3a-a871-4a69-aecd-02c2185b56be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Sep 16 19:16:02.550750 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:02.550711    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Sep 16 19:16:02.551499 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:02.551036    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9" podUID="ac10eb3a-a871-4a69-aecd-02c2185b56be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Sep 16 19:16:03.550642 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:03.550599    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Sep 16 19:16:03.550941 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:03.550905    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9" podUID="ac10eb3a-a871-4a69-aecd-02c2185b56be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Sep 16 19:16:04.550537 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:04.550486    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Sep 16 19:16:04.551204 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:04.551130    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9" podUID="ac10eb3a-a871-4a69-aecd-02c2185b56be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Sep 16 19:16:05.553953 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:05.553922    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:16:05.554373 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:05.554328    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:16:05.554897 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:05.554878    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:16:13.830148 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:13.830115    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"]
Sep 16 19:16:13.833969 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:13.833949    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:13.836579 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:13.836557    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-766919-openshift-default-dockercfg-n2tcn\""
Sep 16 19:16:13.848625 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:13.848595    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"]
Sep 16 19:16:13.933199 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:13.933136    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-credential-socket\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:13.933481 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:13.933457    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-podinfo\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:13.933638 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:13.933623    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-envoy\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:13.933742 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:13.933725    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-744lb\" (UniqueName: \"kubernetes.io/projected/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-kube-api-access-744lb\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:13.933860 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:13.933845    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-data\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:13.934047 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:13.934021    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-token\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:13.934269 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:13.934245    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istiod-ca-cert\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:13.934380 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:13.934328    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-workload-certs\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:13.934453 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:13.934384    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-workload-socket\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.035867 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.035803    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-credential-socket\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.035867 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.035867    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-podinfo\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.036129 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.035917    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-envoy\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.036129 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.035940    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-744lb\" (UniqueName: \"kubernetes.io/projected/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-kube-api-access-744lb\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.036129 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.035964    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-data\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.036129 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.036022    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-token\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.036129 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.036063    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istiod-ca-cert\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.036407 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.036130    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-workload-certs\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.036407 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.036230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-workload-socket\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.036407 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.036281    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-credential-socket\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.036690 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.036661    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-workload-socket\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.036690 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.036684    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-data\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.036868 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.036750    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-workload-certs\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.036989 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.036967    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istiod-ca-cert\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.038698 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.038651    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-podinfo\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.038814 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.038701    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-envoy\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.051986 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.051957    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-token\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.052130 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.052098    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-744lb\" (UniqueName: \"kubernetes.io/projected/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-kube-api-access-744lb\") pod \"e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") " pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.146259 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.146223    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:14.432569 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.432544    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"]
Sep 16 19:16:14.434300 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:16:14.434270    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c0234b_6dab_4ce2_bafb_60a4fcebb1bc.slice/crio-e251fb02f8b58ec21c13f61b303edd629200bfcfd3f2d3ba45c77c492c09e244 WatchSource:0}: Error finding container e251fb02f8b58ec21c13f61b303edd629200bfcfd3f2d3ba45c77c492c09e244: Status 404 returned error can't find the container with id e251fb02f8b58ec21c13f61b303edd629200bfcfd3f2d3ba45c77c492c09e244
Sep 16 19:16:14.436598 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.436564    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:16:14.436679 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.436642    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:16:14.436679 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:14.436671    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:16:15.141649 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:15.141606    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m" event={"ID":"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc","Type":"ContainerStarted","Data":"37971b9ce627b4a3f7434b0d5cbfb74b1a4a535f8602400639c44b02e8fac78e"}
Sep 16 19:16:15.142056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:15.141682    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m" event={"ID":"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc","Type":"ContainerStarted","Data":"e251fb02f8b58ec21c13f61b303edd629200bfcfd3f2d3ba45c77c492c09e244"}
Sep 16 19:16:15.146461 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:15.146432    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:15.151743 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:15.151721    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:15.173536 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:15.173484    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m" podStartSLOduration=2.173468341 podStartE2EDuration="2.173468341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:16:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:16:15.171956664 +0000 UTC m=+2202.959520840" watchObservedRunningTime="2026-09-16 19:16:15.173468341 +0000 UTC m=+2202.961032498"
Sep 16 19:16:16.150245 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:16.150212    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:16.151287 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:16.151267    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:32.191415 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:32.191365    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 16 19:16:32.191415 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:32.191418    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m" podUID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 16 19:16:33.261852 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:33.261749    2578 scope.go:117] "RemoveContainer" containerID="93eb13238043376e5d2666919119dbb2b95ca6506385d0c2598437e381583b66"
Sep 16 19:16:33.267490 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:33.267459    2578 generic.go:356] "Generic (PLEG): container finished" podID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" containerID="37971b9ce627b4a3f7434b0d5cbfb74b1a4a535f8602400639c44b02e8fac78e" exitCode=0
Sep 16 19:16:33.267626 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:33.267537    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m" event={"ID":"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc","Type":"ContainerDied","Data":"37971b9ce627b4a3f7434b0d5cbfb74b1a4a535f8602400639c44b02e8fac78e"}
Sep 16 19:16:33.268140 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:33.268121    2578 scope.go:117] "RemoveContainer" containerID="37971b9ce627b4a3f7434b0d5cbfb74b1a4a535f8602400639c44b02e8fac78e"
Sep 16 19:16:33.271600 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:33.271556    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:16:33.271717 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:33.271643    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:16:33.271717 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:33.271677    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:16:34.276901 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:34.276867    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m" event={"ID":"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc","Type":"ContainerStarted","Data":"5c5a63ad34f9560cf02663310f330416b9a3a6fce9f8818207942e7a4b95e753"}
Sep 16 19:16:35.146607 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:35.146564    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:35.147008 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:35.146960    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Sep 16 19:16:35.147120 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:35.147049    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m" podUID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Sep 16 19:16:36.147992 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:36.147531    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Sep 16 19:16:36.147992 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:36.147605    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m" podUID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Sep 16 19:16:37.147186 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:37.147121    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Sep 16 19:16:37.147398 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:37.147232    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m" podUID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Sep 16 19:16:38.147026 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:38.146975    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Sep 16 19:16:38.147547 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:38.147070    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m" podUID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Sep 16 19:16:39.147718 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:39.147226    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Sep 16 19:16:39.147718 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:39.147317    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m" podUID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Sep 16 19:16:40.150481 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:40.150451    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:40.150883 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:40.150853    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:16:40.151426 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:16:40.151407    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:17:23.785108 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.785073    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"]
Sep 16 19:17:23.788646 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.788626    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.794022 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.793997    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-312c48-openshift-default-dockercfg-clhf2\""
Sep 16 19:17:23.804214 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.804190    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"]
Sep 16 19:17:23.864963 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.864921    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-credential-socket\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.865331 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.865302    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-workload-socket\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.865531 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.865501    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4434810-ce86-415a-a913-fc508a548cfb-istiod-ca-cert\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.865716 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.865699    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-istio-envoy\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.865865 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.865844    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-workload-certs\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.865995 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.865973    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hctgz\" (UniqueName: \"kubernetes.io/projected/b4434810-ce86-415a-a913-fc508a548cfb-kube-api-access-hctgz\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.866158 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.866136    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4434810-ce86-415a-a913-fc508a548cfb-istio-podinfo\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.866338 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.866319    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4434810-ce86-415a-a913-fc508a548cfb-istio-token\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.866441 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.866428    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-istio-data\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.967614 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.967576    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-workload-certs\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.967614 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.967617    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hctgz\" (UniqueName: \"kubernetes.io/projected/b4434810-ce86-415a-a913-fc508a548cfb-kube-api-access-hctgz\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.967844 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.967692    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4434810-ce86-415a-a913-fc508a548cfb-istio-podinfo\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.967844 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.967741    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4434810-ce86-415a-a913-fc508a548cfb-istio-token\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.967844 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.967757    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-istio-data\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.967998 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.967840    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-credential-socket\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.967998 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.967955    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-workload-socket\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.967998 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.967985    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4434810-ce86-415a-a913-fc508a548cfb-istiod-ca-cert\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.968185 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.968046    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-istio-envoy\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.968185 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.968125    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-workload-certs\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.968306 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.968226    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-credential-socket\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.968367 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.968300    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-istio-data\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.968367 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.968326    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-workload-socket\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.968779 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.968755    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4434810-ce86-415a-a913-fc508a548cfb-istiod-ca-cert\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.970355 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.970327    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-istio-envoy\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.970559 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.970541    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4434810-ce86-415a-a913-fc508a548cfb-istio-podinfo\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.976763 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.976735    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4434810-ce86-415a-a913-fc508a548cfb-istio-token\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:23.976894 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:23.976825    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hctgz\" (UniqueName: \"kubernetes.io/projected/b4434810-ce86-415a-a913-fc508a548cfb-kube-api-access-hctgz\") pod \"e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") " pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:24.101592 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:24.101497    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:24.387489 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:24.387463    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"]
Sep 16 19:17:24.389408 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:17:24.389379    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4434810_ce86_415a_a913_fc508a548cfb.slice/crio-0b285e0560355e9e3c0398542d4cfa97f3878f52118c31309cc356d873977144 WatchSource:0}: Error finding container 0b285e0560355e9e3c0398542d4cfa97f3878f52118c31309cc356d873977144: Status 404 returned error can't find the container with id 0b285e0560355e9e3c0398542d4cfa97f3878f52118c31309cc356d873977144
Sep 16 19:17:24.391717 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:24.391681    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:17:24.391833 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:24.391767    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:17:24.391833 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:24.391813    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:17:24.589883 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:24.589841    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k" event={"ID":"b4434810-ce86-415a-a913-fc508a548cfb","Type":"ContainerStarted","Data":"30b85a6e225eb48e0a4a7b16877b98160a49c106cc428c9e0bbdd0ba18db1bc2"}
Sep 16 19:17:24.590089 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:24.589909    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k" event={"ID":"b4434810-ce86-415a-a913-fc508a548cfb","Type":"ContainerStarted","Data":"0b285e0560355e9e3c0398542d4cfa97f3878f52118c31309cc356d873977144"}
Sep 16 19:17:24.616799 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:24.616725    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k" podStartSLOduration=1.616704877 podStartE2EDuration="1.616704877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:17:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:17:24.612030318 +0000 UTC m=+2272.399594500" watchObservedRunningTime="2026-09-16 19:17:24.616704877 +0000 UTC m=+2272.404269033"
Sep 16 19:17:25.101961 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:25.101906    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:25.107334 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:25.107307    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:25.596427 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:25.596396    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:25.597476 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:25.597456    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:17:57.274828 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:57.274795    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr"]
Sep 16 19:17:57.278605 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:57.278580    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr"
Sep 16 19:17:57.281605 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:57.281578    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nmr62\""
Sep 16 19:17:57.290749 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:57.290724    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr"]
Sep 16 19:17:57.418267 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:57.418231    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/818defd5-39e1-44fc-8eaa-a992f44fc6ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr\" (UID: \"818defd5-39e1-44fc-8eaa-a992f44fc6ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr"
Sep 16 19:17:57.519120 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:57.519080    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/818defd5-39e1-44fc-8eaa-a992f44fc6ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr\" (UID: \"818defd5-39e1-44fc-8eaa-a992f44fc6ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr"
Sep 16 19:17:57.519754 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:57.519732    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/818defd5-39e1-44fc-8eaa-a992f44fc6ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr\" (UID: \"818defd5-39e1-44fc-8eaa-a992f44fc6ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr"
Sep 16 19:17:57.590409 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:57.590316    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr"
Sep 16 19:17:57.898044 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:57.898008    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr"]
Sep 16 19:17:57.899719 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:17:57.899696    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod818defd5_39e1_44fc_8eaa_a992f44fc6ad.slice/crio-fff891d440f49a2e0c06117ac4bfc1814072f2507f46cebc37b1c72fa43ed2c3 WatchSource:0}: Error finding container fff891d440f49a2e0c06117ac4bfc1814072f2507f46cebc37b1c72fa43ed2c3: Status 404 returned error can't find the container with id fff891d440f49a2e0c06117ac4bfc1814072f2507f46cebc37b1c72fa43ed2c3
Sep 16 19:17:58.805993 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:58.805951    2578 generic.go:356] "Generic (PLEG): container finished" podID="818defd5-39e1-44fc-8eaa-a992f44fc6ad" containerID="00768b5f921a9a1513e718deb407c6ed1880cefe7789fd34cc28ca361f889ebc" exitCode=0
Sep 16 19:17:58.806420 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:58.806024    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr" event={"ID":"818defd5-39e1-44fc-8eaa-a992f44fc6ad","Type":"ContainerDied","Data":"00768b5f921a9a1513e718deb407c6ed1880cefe7789fd34cc28ca361f889ebc"}
Sep 16 19:17:58.806420 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:58.806089    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr" event={"ID":"818defd5-39e1-44fc-8eaa-a992f44fc6ad","Type":"ContainerStarted","Data":"fff891d440f49a2e0c06117ac4bfc1814072f2507f46cebc37b1c72fa43ed2c3"}
Sep 16 19:17:59.955268 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:17:59.955244    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr"
Sep 16 19:18:00.049577 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:00.049541    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/818defd5-39e1-44fc-8eaa-a992f44fc6ad-maas-api-service-ca\") pod \"818defd5-39e1-44fc-8eaa-a992f44fc6ad\" (UID: \"818defd5-39e1-44fc-8eaa-a992f44fc6ad\") "
Sep 16 19:18:00.049943 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:00.049920    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/818defd5-39e1-44fc-8eaa-a992f44fc6ad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "818defd5-39e1-44fc-8eaa-a992f44fc6ad" (UID: "818defd5-39e1-44fc-8eaa-a992f44fc6ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:18:00.150578 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:00.150543    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/818defd5-39e1-44fc-8eaa-a992f44fc6ad-maas-api-service-ca\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:18:00.820702 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:00.820658    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr" event={"ID":"818defd5-39e1-44fc-8eaa-a992f44fc6ad","Type":"ContainerDied","Data":"fff891d440f49a2e0c06117ac4bfc1814072f2507f46cebc37b1c72fa43ed2c3"}
Sep 16 19:18:00.820962 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:00.820946    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fff891d440f49a2e0c06117ac4bfc1814072f2507f46cebc37b1c72fa43ed2c3"
Sep 16 19:18:00.821119 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:00.821107    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr"
Sep 16 19:18:01.828125 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:01.828093    2578 generic.go:356] "Generic (PLEG): container finished" podID="b4434810-ce86-415a-a913-fc508a548cfb" containerID="30b85a6e225eb48e0a4a7b16877b98160a49c106cc428c9e0bbdd0ba18db1bc2" exitCode=0
Sep 16 19:18:01.828690 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:01.828183    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k" event={"ID":"b4434810-ce86-415a-a913-fc508a548cfb","Type":"ContainerDied","Data":"30b85a6e225eb48e0a4a7b16877b98160a49c106cc428c9e0bbdd0ba18db1bc2"}
Sep 16 19:18:01.828756 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:01.828742    2578 scope.go:117] "RemoveContainer" containerID="30b85a6e225eb48e0a4a7b16877b98160a49c106cc428c9e0bbdd0ba18db1bc2"
Sep 16 19:18:01.829960 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:01.829925    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:18:01.830056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:01.829998    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:18:01.830056 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:01.830033    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 16 19:18:02.836522 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:02.836485    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k" event={"ID":"b4434810-ce86-415a-a913-fc508a548cfb","Type":"ContainerStarted","Data":"5738e99b34c7e533ab4fb75b08f54cd8731dd20c6133d92cd10bb1ac99b0092f"}
Sep 16 19:18:03.101912 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:03.101804    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:18:03.102103 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:03.102061    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Sep 16 19:18:03.102224 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:03.102117    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k" podUID="b4434810-ce86-415a-a913-fc508a548cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Sep 16 19:18:04.102783 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:04.102747    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Sep 16 19:18:04.103157 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:04.102805    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k" podUID="b4434810-ce86-415a-a913-fc508a548cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Sep 16 19:18:05.102536 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:05.102498    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Sep 16 19:18:05.102735 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:05.102555    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k" podUID="b4434810-ce86-415a-a913-fc508a548cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Sep 16 19:18:06.102854 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:06.102809    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Sep 16 19:18:06.103367 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:06.102880    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k" podUID="b4434810-ce86-415a-a913-fc508a548cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Sep 16 19:18:07.102649 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:07.102610    2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Sep 16 19:18:07.102831 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:07.102672    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k" podUID="b4434810-ce86-415a-a913-fc508a548cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Sep 16 19:18:08.106787 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:08.106752    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:18:08.107163 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:08.107015    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:18:08.107862 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:08.107846    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:18:08.775133 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:08.775081    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr"]
Sep 16 19:18:08.777886 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:08.777857    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-312c48-79bqr"]
Sep 16 19:18:10.709996 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:10.709966    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="818defd5-39e1-44fc-8eaa-a992f44fc6ad" path="/var/lib/kubelet/pods/818defd5-39e1-44fc-8eaa-a992f44fc6ad/volumes"
Sep 16 19:18:16.163974 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:16.163902    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"]
Sep 16 19:18:16.164492 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:16.164311    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k" podUID="b4434810-ce86-415a-a913-fc508a548cfb" containerName="istio-proxy" containerID="cri-o://5738e99b34c7e533ab4fb75b08f54cd8731dd20c6133d92cd10bb1ac99b0092f" gracePeriod=30
Sep 16 19:18:21.413317 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.413291    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:18:21.465754 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.465672    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4434810-ce86-415a-a913-fc508a548cfb-istiod-ca-cert\") pod \"b4434810-ce86-415a-a913-fc508a548cfb\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") "
Sep 16 19:18:21.465754 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.465718    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hctgz\" (UniqueName: \"kubernetes.io/projected/b4434810-ce86-415a-a913-fc508a548cfb-kube-api-access-hctgz\") pod \"b4434810-ce86-415a-a913-fc508a548cfb\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") "
Sep 16 19:18:21.465754 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.465751    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-istio-data\") pod \"b4434810-ce86-415a-a913-fc508a548cfb\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") "
Sep 16 19:18:21.466052 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.465781    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-istio-envoy\") pod \"b4434810-ce86-415a-a913-fc508a548cfb\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") "
Sep 16 19:18:21.466052 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.465841    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4434810-ce86-415a-a913-fc508a548cfb-istio-token\") pod \"b4434810-ce86-415a-a913-fc508a548cfb\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") "
Sep 16 19:18:21.466052 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.465866    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-credential-socket\") pod \"b4434810-ce86-415a-a913-fc508a548cfb\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") "
Sep 16 19:18:21.466052 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.465918    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-workload-certs\") pod \"b4434810-ce86-415a-a913-fc508a548cfb\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") "
Sep 16 19:18:21.466052 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.465991    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4434810-ce86-415a-a913-fc508a548cfb-istio-podinfo\") pod \"b4434810-ce86-415a-a913-fc508a548cfb\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") "
Sep 16 19:18:21.466052 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.466037    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-workload-socket\") pod \"b4434810-ce86-415a-a913-fc508a548cfb\" (UID: \"b4434810-ce86-415a-a913-fc508a548cfb\") "
Sep 16 19:18:21.466397 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.466090    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-istio-data" (OuterVolumeSpecName: "istio-data") pod "b4434810-ce86-415a-a913-fc508a548cfb" (UID: "b4434810-ce86-415a-a913-fc508a548cfb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:18:21.466397 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.466155    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4434810-ce86-415a-a913-fc508a548cfb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b4434810-ce86-415a-a913-fc508a548cfb" (UID: "b4434810-ce86-415a-a913-fc508a548cfb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:18:21.466524 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.466402    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b4434810-ce86-415a-a913-fc508a548cfb" (UID: "b4434810-ce86-415a-a913-fc508a548cfb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:18:21.466524 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.466432    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b4434810-ce86-415a-a913-fc508a548cfb" (UID: "b4434810-ce86-415a-a913-fc508a548cfb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:18:21.466636 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.466587    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:18:21.466636 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.466608    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:18:21.466636 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.466622    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:18:21.466793 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.466635    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4434810-ce86-415a-a913-fc508a548cfb-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:18:21.466793 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.466697    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b4434810-ce86-415a-a913-fc508a548cfb" (UID: "b4434810-ce86-415a-a913-fc508a548cfb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:18:21.468343 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.468312    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4434810-ce86-415a-a913-fc508a548cfb-istio-token" (OuterVolumeSpecName: "istio-token") pod "b4434810-ce86-415a-a913-fc508a548cfb" (UID: "b4434810-ce86-415a-a913-fc508a548cfb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:18:21.468455 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.468340    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b4434810-ce86-415a-a913-fc508a548cfb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b4434810-ce86-415a-a913-fc508a548cfb" (UID: "b4434810-ce86-415a-a913-fc508a548cfb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:18:21.468455 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.468346    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b4434810-ce86-415a-a913-fc508a548cfb" (UID: "b4434810-ce86-415a-a913-fc508a548cfb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:18:21.468757 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.468742    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4434810-ce86-415a-a913-fc508a548cfb-kube-api-access-hctgz" (OuterVolumeSpecName: "kube-api-access-hctgz") pod "b4434810-ce86-415a-a913-fc508a548cfb" (UID: "b4434810-ce86-415a-a913-fc508a548cfb"). InnerVolumeSpecName "kube-api-access-hctgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:18:21.568224 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.568153    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4434810-ce86-415a-a913-fc508a548cfb-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:18:21.568224 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.568221    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:18:21.568224 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.568232    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4434810-ce86-415a-a913-fc508a548cfb-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:18:21.568468 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.568243    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hctgz\" (UniqueName: \"kubernetes.io/projected/b4434810-ce86-415a-a913-fc508a548cfb-kube-api-access-hctgz\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:18:21.568468 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.568253    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4434810-ce86-415a-a913-fc508a548cfb-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:18:21.975579 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.975539    2578 generic.go:356] "Generic (PLEG): container finished" podID="b4434810-ce86-415a-a913-fc508a548cfb" containerID="5738e99b34c7e533ab4fb75b08f54cd8731dd20c6133d92cd10bb1ac99b0092f" exitCode=0
Sep 16 19:18:21.975763 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.975592    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"
Sep 16 19:18:21.975763 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.975613    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k" event={"ID":"b4434810-ce86-415a-a913-fc508a548cfb","Type":"ContainerDied","Data":"5738e99b34c7e533ab4fb75b08f54cd8731dd20c6133d92cd10bb1ac99b0092f"}
Sep 16 19:18:21.975763 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.975670    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k" event={"ID":"b4434810-ce86-415a-a913-fc508a548cfb","Type":"ContainerDied","Data":"0b285e0560355e9e3c0398542d4cfa97f3878f52118c31309cc356d873977144"}
Sep 16 19:18:21.975763 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.975693    2578 scope.go:117] "RemoveContainer" containerID="5738e99b34c7e533ab4fb75b08f54cd8731dd20c6133d92cd10bb1ac99b0092f"
Sep 16 19:18:21.988573 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.987075    2578 scope.go:117] "RemoveContainer" containerID="30b85a6e225eb48e0a4a7b16877b98160a49c106cc428c9e0bbdd0ba18db1bc2"
Sep 16 19:18:21.997708 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.997689    2578 scope.go:117] "RemoveContainer" containerID="5738e99b34c7e533ab4fb75b08f54cd8731dd20c6133d92cd10bb1ac99b0092f"
Sep 16 19:18:21.997999 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:18:21.997979    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5738e99b34c7e533ab4fb75b08f54cd8731dd20c6133d92cd10bb1ac99b0092f\": container with ID starting with 5738e99b34c7e533ab4fb75b08f54cd8731dd20c6133d92cd10bb1ac99b0092f not found: ID does not exist" containerID="5738e99b34c7e533ab4fb75b08f54cd8731dd20c6133d92cd10bb1ac99b0092f"
Sep 16 19:18:21.998077 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.998013    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5738e99b34c7e533ab4fb75b08f54cd8731dd20c6133d92cd10bb1ac99b0092f"} err="failed to get container status \"5738e99b34c7e533ab4fb75b08f54cd8731dd20c6133d92cd10bb1ac99b0092f\": rpc error: code = NotFound desc = could not find container \"5738e99b34c7e533ab4fb75b08f54cd8731dd20c6133d92cd10bb1ac99b0092f\": container with ID starting with 5738e99b34c7e533ab4fb75b08f54cd8731dd20c6133d92cd10bb1ac99b0092f not found: ID does not exist"
Sep 16 19:18:21.998077 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.998040    2578 scope.go:117] "RemoveContainer" containerID="30b85a6e225eb48e0a4a7b16877b98160a49c106cc428c9e0bbdd0ba18db1bc2"
Sep 16 19:18:21.998324 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:18:21.998306    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30b85a6e225eb48e0a4a7b16877b98160a49c106cc428c9e0bbdd0ba18db1bc2\": container with ID starting with 30b85a6e225eb48e0a4a7b16877b98160a49c106cc428c9e0bbdd0ba18db1bc2 not found: ID does not exist" containerID="30b85a6e225eb48e0a4a7b16877b98160a49c106cc428c9e0bbdd0ba18db1bc2"
Sep 16 19:18:21.998371 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:21.998330    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30b85a6e225eb48e0a4a7b16877b98160a49c106cc428c9e0bbdd0ba18db1bc2"} err="failed to get container status \"30b85a6e225eb48e0a4a7b16877b98160a49c106cc428c9e0bbdd0ba18db1bc2\": rpc error: code = NotFound desc = could not find container \"30b85a6e225eb48e0a4a7b16877b98160a49c106cc428c9e0bbdd0ba18db1bc2\": container with ID starting with 30b85a6e225eb48e0a4a7b16877b98160a49c106cc428c9e0bbdd0ba18db1bc2 not found: ID does not exist"
Sep 16 19:18:22.012402 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:22.012377    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"]
Sep 16 19:18:22.015067 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:22.015047    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-312c48-openshift-default-5dc96966c4-slg5k"]
Sep 16 19:18:22.711550 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:18:22.711520    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4434810-ce86-415a-a913-fc508a548cfb" path="/var/lib/kubelet/pods/b4434810-ce86-415a-a913-fc508a548cfb/volumes"
Sep 16 19:19:32.873972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:32.873932    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"]
Sep 16 19:19:32.874509 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:32.874135    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9" podUID="ac10eb3a-a871-4a69-aecd-02c2185b56be" containerName="istio-proxy" containerID="cri-o://2e380695909c069d3c144ae884ec6e216aafd7109cc53495b16a4eeacc8156a4" gracePeriod=30
Sep 16 19:19:32.941152 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:32.941094    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 19:19:32.953883 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:32.953855    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 19:19:32.955817 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:32.955791    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 19:19:32.964989 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:32.964960    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 19:19:35.556677 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:35.556636    2578 patch_prober.go:28] interesting pod/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 19:19:35.557058 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:35.556709    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9" podUID="ac10eb3a-a871-4a69-aecd-02c2185b56be" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:19:38.136774 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.136708    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:19:38.257295 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.257258    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-envoy\") pod \"ac10eb3a-a871-4a69-aecd-02c2185b56be\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") "
Sep 16 19:19:38.257295 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.257301    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-workload-socket\") pod \"ac10eb3a-a871-4a69-aecd-02c2185b56be\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") "
Sep 16 19:19:38.257548 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.257349    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-credential-socket\") pod \"ac10eb3a-a871-4a69-aecd-02c2185b56be\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") "
Sep 16 19:19:38.257548 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.257393    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-podinfo\") pod \"ac10eb3a-a871-4a69-aecd-02c2185b56be\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") "
Sep 16 19:19:38.257548 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.257438    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-token\") pod \"ac10eb3a-a871-4a69-aecd-02c2185b56be\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") "
Sep 16 19:19:38.257548 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.257481    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-workload-certs\") pod \"ac10eb3a-a871-4a69-aecd-02c2185b56be\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") "
Sep 16 19:19:38.257548 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.257521    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-data\") pod \"ac10eb3a-a871-4a69-aecd-02c2185b56be\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") "
Sep 16 19:19:38.257823 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.257555    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac10eb3a-a871-4a69-aecd-02c2185b56be-istiod-ca-cert\") pod \"ac10eb3a-a871-4a69-aecd-02c2185b56be\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") "
Sep 16 19:19:38.257823 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.257577    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzddv\" (UniqueName: \"kubernetes.io/projected/ac10eb3a-a871-4a69-aecd-02c2185b56be-kube-api-access-fzddv\") pod \"ac10eb3a-a871-4a69-aecd-02c2185b56be\" (UID: \"ac10eb3a-a871-4a69-aecd-02c2185b56be\") "
Sep 16 19:19:38.257823 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.257605    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ac10eb3a-a871-4a69-aecd-02c2185b56be" (UID: "ac10eb3a-a871-4a69-aecd-02c2185b56be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:19:38.257823 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.257675    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ac10eb3a-a871-4a69-aecd-02c2185b56be" (UID: "ac10eb3a-a871-4a69-aecd-02c2185b56be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:19:38.258023 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.257872    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ac10eb3a-a871-4a69-aecd-02c2185b56be" (UID: "ac10eb3a-a871-4a69-aecd-02c2185b56be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:19:38.258113 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.258089    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac10eb3a-a871-4a69-aecd-02c2185b56be-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ac10eb3a-a871-4a69-aecd-02c2185b56be" (UID: "ac10eb3a-a871-4a69-aecd-02c2185b56be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:19:38.258200 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.258128    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-data" (OuterVolumeSpecName: "istio-data") pod "ac10eb3a-a871-4a69-aecd-02c2185b56be" (UID: "ac10eb3a-a871-4a69-aecd-02c2185b56be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:19:38.258278 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.258259    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:38.258339 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.258284    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:38.258339 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.258299    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:38.259745 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.259714    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ac10eb3a-a871-4a69-aecd-02c2185b56be" (UID: "ac10eb3a-a871-4a69-aecd-02c2185b56be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:19:38.259888 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.259787    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac10eb3a-a871-4a69-aecd-02c2185b56be-kube-api-access-fzddv" (OuterVolumeSpecName: "kube-api-access-fzddv") pod "ac10eb3a-a871-4a69-aecd-02c2185b56be" (UID: "ac10eb3a-a871-4a69-aecd-02c2185b56be"). InnerVolumeSpecName "kube-api-access-fzddv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:19:38.259888 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.259817    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ac10eb3a-a871-4a69-aecd-02c2185b56be" (UID: "ac10eb3a-a871-4a69-aecd-02c2185b56be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:19:38.259979 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.259933    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-token" (OuterVolumeSpecName: "istio-token") pod "ac10eb3a-a871-4a69-aecd-02c2185b56be" (UID: "ac10eb3a-a871-4a69-aecd-02c2185b56be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:19:38.359722 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.359688    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac10eb3a-a871-4a69-aecd-02c2185b56be-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:38.359722 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.359716    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzddv\" (UniqueName: \"kubernetes.io/projected/ac10eb3a-a871-4a69-aecd-02c2185b56be-kube-api-access-fzddv\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:38.359722 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.359726    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:38.359960 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.359735    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:38.359960 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.359744    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:38.359960 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.359752    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac10eb3a-a871-4a69-aecd-02c2185b56be-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:38.461523 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.461426    2578 generic.go:356] "Generic (PLEG): container finished" podID="ac10eb3a-a871-4a69-aecd-02c2185b56be" containerID="2e380695909c069d3c144ae884ec6e216aafd7109cc53495b16a4eeacc8156a4" exitCode=0
Sep 16 19:19:38.461523 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.461487    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"
Sep 16 19:19:38.461523 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.461507    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9" event={"ID":"ac10eb3a-a871-4a69-aecd-02c2185b56be","Type":"ContainerDied","Data":"2e380695909c069d3c144ae884ec6e216aafd7109cc53495b16a4eeacc8156a4"}
Sep 16 19:19:38.461806 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.461563    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9" event={"ID":"ac10eb3a-a871-4a69-aecd-02c2185b56be","Type":"ContainerDied","Data":"99be4a91ce9c935165468166b2cd9f8dedf2420c924bac214bf81e5339c9cfc4"}
Sep 16 19:19:38.461806 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.461586    2578 scope.go:117] "RemoveContainer" containerID="2e380695909c069d3c144ae884ec6e216aafd7109cc53495b16a4eeacc8156a4"
Sep 16 19:19:38.471862 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.471844    2578 scope.go:117] "RemoveContainer" containerID="335934a5879d66efe9d0ae862a6a51c37c82936c1173aa23c122284104d11bf5"
Sep 16 19:19:38.480648 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.480616    2578 scope.go:117] "RemoveContainer" containerID="2e380695909c069d3c144ae884ec6e216aafd7109cc53495b16a4eeacc8156a4"
Sep 16 19:19:38.480931 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:19:38.480911    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e380695909c069d3c144ae884ec6e216aafd7109cc53495b16a4eeacc8156a4\": container with ID starting with 2e380695909c069d3c144ae884ec6e216aafd7109cc53495b16a4eeacc8156a4 not found: ID does not exist" containerID="2e380695909c069d3c144ae884ec6e216aafd7109cc53495b16a4eeacc8156a4"
Sep 16 19:19:38.480983 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.480941    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e380695909c069d3c144ae884ec6e216aafd7109cc53495b16a4eeacc8156a4"} err="failed to get container status \"2e380695909c069d3c144ae884ec6e216aafd7109cc53495b16a4eeacc8156a4\": rpc error: code = NotFound desc = could not find container \"2e380695909c069d3c144ae884ec6e216aafd7109cc53495b16a4eeacc8156a4\": container with ID starting with 2e380695909c069d3c144ae884ec6e216aafd7109cc53495b16a4eeacc8156a4 not found: ID does not exist"
Sep 16 19:19:38.480983 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.480958    2578 scope.go:117] "RemoveContainer" containerID="335934a5879d66efe9d0ae862a6a51c37c82936c1173aa23c122284104d11bf5"
Sep 16 19:19:38.481240 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:19:38.481222    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"335934a5879d66efe9d0ae862a6a51c37c82936c1173aa23c122284104d11bf5\": container with ID starting with 335934a5879d66efe9d0ae862a6a51c37c82936c1173aa23c122284104d11bf5 not found: ID does not exist" containerID="335934a5879d66efe9d0ae862a6a51c37c82936c1173aa23c122284104d11bf5"
Sep 16 19:19:38.481296 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.481246    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"335934a5879d66efe9d0ae862a6a51c37c82936c1173aa23c122284104d11bf5"} err="failed to get container status \"335934a5879d66efe9d0ae862a6a51c37c82936c1173aa23c122284104d11bf5\": rpc error: code = NotFound desc = could not find container \"335934a5879d66efe9d0ae862a6a51c37c82936c1173aa23c122284104d11bf5\": container with ID starting with 335934a5879d66efe9d0ae862a6a51c37c82936c1173aa23c122284104d11bf5 not found: ID does not exist"
Sep 16 19:19:38.487996 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.487979    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"]
Sep 16 19:19:38.494989 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.494968    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-4992ef-openshift-default-667df6bd9b-dm7p9"]
Sep 16 19:19:38.709576 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:38.709545    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac10eb3a-a871-4a69-aecd-02c2185b56be" path="/var/lib/kubelet/pods/ac10eb3a-a871-4a69-aecd-02c2185b56be/volumes"
Sep 16 19:19:51.262004 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:51.261964    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"]
Sep 16 19:19:51.262544 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:51.262232    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m" podUID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" containerName="istio-proxy" containerID="cri-o://5c5a63ad34f9560cf02663310f330416b9a3a6fce9f8818207942e7a4b95e753" gracePeriod=30
Sep 16 19:19:55.153914 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:55.153836    2578 patch_prober.go:28] interesting pod/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 19:19:55.154335 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:55.153907    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m" podUID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:19:56.514889 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.514860    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:19:56.542943 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.542853    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istiod-ca-cert\") pod \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") "
Sep 16 19:19:56.542943 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.542902    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-workload-socket\") pod \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") "
Sep 16 19:19:56.543158 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.542954    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-podinfo\") pod \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") "
Sep 16 19:19:56.543158 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.543014    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-data\") pod \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") "
Sep 16 19:19:56.543158 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.543056    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-workload-certs\") pod \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") "
Sep 16 19:19:56.543158 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.543105    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-envoy\") pod \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") "
Sep 16 19:19:56.543415 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.543168    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-token\") pod \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") "
Sep 16 19:19:56.543415 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.543218    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-744lb\" (UniqueName: \"kubernetes.io/projected/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-kube-api-access-744lb\") pod \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") "
Sep 16 19:19:56.543415 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.543245    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-credential-socket\") pod \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\" (UID: \"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc\") "
Sep 16 19:19:56.543415 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.543335    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" (UID: "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:19:56.543415 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.543374    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-data" (OuterVolumeSpecName: "istio-data") pod "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" (UID: "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:19:56.543680 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.543595    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" (UID: "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:19:56.544274 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.543774    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" (UID: "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:19:56.544784 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.544695    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:56.544784 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.544727    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:56.544784 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.544741    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:56.544784 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.544754    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:56.546066 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.543626    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" (UID: "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:19:56.547513 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.546875    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" (UID: "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:19:56.547513 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.546963    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-kube-api-access-744lb" (OuterVolumeSpecName: "kube-api-access-744lb") pod "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" (UID: "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc"). InnerVolumeSpecName "kube-api-access-744lb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:19:56.547859 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.547833    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" (UID: "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:19:56.548219 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.548193    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-token" (OuterVolumeSpecName: "istio-token") pod "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" (UID: "53c0234b-6dab-4ce2-bafb-60a4fcebb1bc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:19:56.583035 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.582998    2578 generic.go:356] "Generic (PLEG): container finished" podID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" containerID="5c5a63ad34f9560cf02663310f330416b9a3a6fce9f8818207942e7a4b95e753" exitCode=0
Sep 16 19:19:56.583209 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.583051    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"
Sep 16 19:19:56.583209 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.583074    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m" event={"ID":"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc","Type":"ContainerDied","Data":"5c5a63ad34f9560cf02663310f330416b9a3a6fce9f8818207942e7a4b95e753"}
Sep 16 19:19:56.583209 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.583131    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m" event={"ID":"53c0234b-6dab-4ce2-bafb-60a4fcebb1bc","Type":"ContainerDied","Data":"e251fb02f8b58ec21c13f61b303edd629200bfcfd3f2d3ba45c77c492c09e244"}
Sep 16 19:19:56.583209 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.583154    2578 scope.go:117] "RemoveContainer" containerID="5c5a63ad34f9560cf02663310f330416b9a3a6fce9f8818207942e7a4b95e753"
Sep 16 19:19:56.593214 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.593159    2578 scope.go:117] "RemoveContainer" containerID="37971b9ce627b4a3f7434b0d5cbfb74b1a4a535f8602400639c44b02e8fac78e"
Sep 16 19:19:56.602317 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.602300    2578 scope.go:117] "RemoveContainer" containerID="5c5a63ad34f9560cf02663310f330416b9a3a6fce9f8818207942e7a4b95e753"
Sep 16 19:19:56.602591 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:19:56.602570    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c5a63ad34f9560cf02663310f330416b9a3a6fce9f8818207942e7a4b95e753\": container with ID starting with 5c5a63ad34f9560cf02663310f330416b9a3a6fce9f8818207942e7a4b95e753 not found: ID does not exist" containerID="5c5a63ad34f9560cf02663310f330416b9a3a6fce9f8818207942e7a4b95e753"
Sep 16 19:19:56.602635 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.602599    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c5a63ad34f9560cf02663310f330416b9a3a6fce9f8818207942e7a4b95e753"} err="failed to get container status \"5c5a63ad34f9560cf02663310f330416b9a3a6fce9f8818207942e7a4b95e753\": rpc error: code = NotFound desc = could not find container \"5c5a63ad34f9560cf02663310f330416b9a3a6fce9f8818207942e7a4b95e753\": container with ID starting with 5c5a63ad34f9560cf02663310f330416b9a3a6fce9f8818207942e7a4b95e753 not found: ID does not exist"
Sep 16 19:19:56.602635 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.602617    2578 scope.go:117] "RemoveContainer" containerID="37971b9ce627b4a3f7434b0d5cbfb74b1a4a535f8602400639c44b02e8fac78e"
Sep 16 19:19:56.602876 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:19:56.602860    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37971b9ce627b4a3f7434b0d5cbfb74b1a4a535f8602400639c44b02e8fac78e\": container with ID starting with 37971b9ce627b4a3f7434b0d5cbfb74b1a4a535f8602400639c44b02e8fac78e not found: ID does not exist" containerID="37971b9ce627b4a3f7434b0d5cbfb74b1a4a535f8602400639c44b02e8fac78e"
Sep 16 19:19:56.602935 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.602878    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37971b9ce627b4a3f7434b0d5cbfb74b1a4a535f8602400639c44b02e8fac78e"} err="failed to get container status \"37971b9ce627b4a3f7434b0d5cbfb74b1a4a535f8602400639c44b02e8fac78e\": rpc error: code = NotFound desc = could not find container \"37971b9ce627b4a3f7434b0d5cbfb74b1a4a535f8602400639c44b02e8fac78e\": container with ID starting with 37971b9ce627b4a3f7434b0d5cbfb74b1a4a535f8602400639c44b02e8fac78e not found: ID does not exist"
Sep 16 19:19:56.619119 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.619095    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"]
Sep 16 19:19:56.623547 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.623525    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-766919-openshift-default-7dcc6d77fb-wc24m"]
Sep 16 19:19:56.645634 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.645613    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:56.645634 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.645636    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-744lb\" (UniqueName: \"kubernetes.io/projected/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-kube-api-access-744lb\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:56.645769 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.645646    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:56.645769 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.645659    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:56.645769 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.645669    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:19:56.709560 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:19:56.709527    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" path="/var/lib/kubelet/pods/53c0234b-6dab-4ce2-bafb-60a4fcebb1bc/volumes"
Sep 16 19:20:09.934522 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:09.934486    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"]
Sep 16 19:20:09.935067 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:09.934731    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r" podUID="a223baca-28f9-49d2-bd57-798b4350d102" containerName="istio-proxy" containerID="cri-o://53a3adc80a31ea2115cb82e785bcf3a34cc8bf1f07e5edd330f55d8ce5ee48f4" gracePeriod=30
Sep 16 19:20:15.186100 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.186074    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:20:15.338212 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.338086    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-workload-certs\") pod \"a223baca-28f9-49d2-bd57-798b4350d102\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") "
Sep 16 19:20:15.338212 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.338120    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-istio-envoy\") pod \"a223baca-28f9-49d2-bd57-798b4350d102\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") "
Sep 16 19:20:15.338212 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.338158    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f8zc\" (UniqueName: \"kubernetes.io/projected/a223baca-28f9-49d2-bd57-798b4350d102-kube-api-access-4f8zc\") pod \"a223baca-28f9-49d2-bd57-798b4350d102\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") "
Sep 16 19:20:15.338504 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.338231    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-istio-data\") pod \"a223baca-28f9-49d2-bd57-798b4350d102\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") "
Sep 16 19:20:15.338504 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.338255    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a223baca-28f9-49d2-bd57-798b4350d102-istiod-ca-cert\") pod \"a223baca-28f9-49d2-bd57-798b4350d102\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") "
Sep 16 19:20:15.338504 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.338339    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-workload-socket\") pod \"a223baca-28f9-49d2-bd57-798b4350d102\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") "
Sep 16 19:20:15.338504 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.338386    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a223baca-28f9-49d2-bd57-798b4350d102-istio-token\") pod \"a223baca-28f9-49d2-bd57-798b4350d102\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") "
Sep 16 19:20:15.338504 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.338412    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a223baca-28f9-49d2-bd57-798b4350d102-istio-podinfo\") pod \"a223baca-28f9-49d2-bd57-798b4350d102\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") "
Sep 16 19:20:15.338504 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.338480    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-credential-socket\") pod \"a223baca-28f9-49d2-bd57-798b4350d102\" (UID: \"a223baca-28f9-49d2-bd57-798b4350d102\") "
Sep 16 19:20:15.338826 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.338473    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a223baca-28f9-49d2-bd57-798b4350d102" (UID: "a223baca-28f9-49d2-bd57-798b4350d102"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:20:15.338882 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.338576    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-istio-data" (OuterVolumeSpecName: "istio-data") pod "a223baca-28f9-49d2-bd57-798b4350d102" (UID: "a223baca-28f9-49d2-bd57-798b4350d102"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:20:15.338882 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.338726    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a223baca-28f9-49d2-bd57-798b4350d102-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a223baca-28f9-49d2-bd57-798b4350d102" (UID: "a223baca-28f9-49d2-bd57-798b4350d102"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:20:15.338980 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.338755    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a223baca-28f9-49d2-bd57-798b4350d102" (UID: "a223baca-28f9-49d2-bd57-798b4350d102"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:20:15.338980 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.338938    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a223baca-28f9-49d2-bd57-798b4350d102" (UID: "a223baca-28f9-49d2-bd57-798b4350d102"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:20:15.340536 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.340501    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a223baca-28f9-49d2-bd57-798b4350d102-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a223baca-28f9-49d2-bd57-798b4350d102" (UID: "a223baca-28f9-49d2-bd57-798b4350d102"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:20:15.340649 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.340567    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a223baca-28f9-49d2-bd57-798b4350d102-kube-api-access-4f8zc" (OuterVolumeSpecName: "kube-api-access-4f8zc") pod "a223baca-28f9-49d2-bd57-798b4350d102" (UID: "a223baca-28f9-49d2-bd57-798b4350d102"). InnerVolumeSpecName "kube-api-access-4f8zc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:20:15.340858 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.340831    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a223baca-28f9-49d2-bd57-798b4350d102" (UID: "a223baca-28f9-49d2-bd57-798b4350d102"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:20:15.340858 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.340842    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a223baca-28f9-49d2-bd57-798b4350d102-istio-token" (OuterVolumeSpecName: "istio-token") pod "a223baca-28f9-49d2-bd57-798b4350d102" (UID: "a223baca-28f9-49d2-bd57-798b4350d102"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:20:15.439850 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.439814    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:15.439850 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.439846    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a223baca-28f9-49d2-bd57-798b4350d102-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:15.439850 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.439855    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a223baca-28f9-49d2-bd57-798b4350d102-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:15.439850 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.439863    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:15.440118 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.439871    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:15.440118 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.439879    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:15.440118 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.439887    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f8zc\" (UniqueName: \"kubernetes.io/projected/a223baca-28f9-49d2-bd57-798b4350d102-kube-api-access-4f8zc\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:15.440118 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.439895    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a223baca-28f9-49d2-bd57-798b4350d102-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:15.440118 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.439924    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a223baca-28f9-49d2-bd57-798b4350d102-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:15.711904 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.711872    2578 generic.go:356] "Generic (PLEG): container finished" podID="a223baca-28f9-49d2-bd57-798b4350d102" containerID="53a3adc80a31ea2115cb82e785bcf3a34cc8bf1f07e5edd330f55d8ce5ee48f4" exitCode=0
Sep 16 19:20:15.712125 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.711924    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"
Sep 16 19:20:15.712125 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.711959    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r" event={"ID":"a223baca-28f9-49d2-bd57-798b4350d102","Type":"ContainerDied","Data":"53a3adc80a31ea2115cb82e785bcf3a34cc8bf1f07e5edd330f55d8ce5ee48f4"}
Sep 16 19:20:15.712125 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.712018    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r" event={"ID":"a223baca-28f9-49d2-bd57-798b4350d102","Type":"ContainerDied","Data":"61caea3597b3e24432c443e44a40c2565ac208ce9b46ab212a54c9bdb75be9ee"}
Sep 16 19:20:15.712125 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.712048    2578 scope.go:117] "RemoveContainer" containerID="53a3adc80a31ea2115cb82e785bcf3a34cc8bf1f07e5edd330f55d8ce5ee48f4"
Sep 16 19:20:15.722277 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.722258    2578 scope.go:117] "RemoveContainer" containerID="de976220f70c568f2cb05275fb6abf599190a1dafdbe789c248b07cfb66c9777"
Sep 16 19:20:15.731470 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.731452    2578 scope.go:117] "RemoveContainer" containerID="53a3adc80a31ea2115cb82e785bcf3a34cc8bf1f07e5edd330f55d8ce5ee48f4"
Sep 16 19:20:15.731753 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:20:15.731733    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53a3adc80a31ea2115cb82e785bcf3a34cc8bf1f07e5edd330f55d8ce5ee48f4\": container with ID starting with 53a3adc80a31ea2115cb82e785bcf3a34cc8bf1f07e5edd330f55d8ce5ee48f4 not found: ID does not exist" containerID="53a3adc80a31ea2115cb82e785bcf3a34cc8bf1f07e5edd330f55d8ce5ee48f4"
Sep 16 19:20:15.731821 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.731760    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53a3adc80a31ea2115cb82e785bcf3a34cc8bf1f07e5edd330f55d8ce5ee48f4"} err="failed to get container status \"53a3adc80a31ea2115cb82e785bcf3a34cc8bf1f07e5edd330f55d8ce5ee48f4\": rpc error: code = NotFound desc = could not find container \"53a3adc80a31ea2115cb82e785bcf3a34cc8bf1f07e5edd330f55d8ce5ee48f4\": container with ID starting with 53a3adc80a31ea2115cb82e785bcf3a34cc8bf1f07e5edd330f55d8ce5ee48f4 not found: ID does not exist"
Sep 16 19:20:15.731821 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.731776    2578 scope.go:117] "RemoveContainer" containerID="de976220f70c568f2cb05275fb6abf599190a1dafdbe789c248b07cfb66c9777"
Sep 16 19:20:15.732061 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:20:15.732043    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de976220f70c568f2cb05275fb6abf599190a1dafdbe789c248b07cfb66c9777\": container with ID starting with de976220f70c568f2cb05275fb6abf599190a1dafdbe789c248b07cfb66c9777 not found: ID does not exist" containerID="de976220f70c568f2cb05275fb6abf599190a1dafdbe789c248b07cfb66c9777"
Sep 16 19:20:15.732110 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.732068    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de976220f70c568f2cb05275fb6abf599190a1dafdbe789c248b07cfb66c9777"} err="failed to get container status \"de976220f70c568f2cb05275fb6abf599190a1dafdbe789c248b07cfb66c9777\": rpc error: code = NotFound desc = could not find container \"de976220f70c568f2cb05275fb6abf599190a1dafdbe789c248b07cfb66c9777\": container with ID starting with de976220f70c568f2cb05275fb6abf599190a1dafdbe789c248b07cfb66c9777 not found: ID does not exist"
Sep 16 19:20:15.736221 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.736197    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"]
Sep 16 19:20:15.740029 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:15.740009    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ca3f9a-openshift-default-b646c6b94-4gc7r"]
Sep 16 19:20:16.710419 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:16.710384    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a223baca-28f9-49d2-bd57-798b4350d102" path="/var/lib/kubelet/pods/a223baca-28f9-49d2-bd57-798b4350d102/volumes"
Sep 16 19:20:29.705290 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:29.705249    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"]
Sep 16 19:20:29.706405 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:29.705510    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql" podUID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" containerName="istio-proxy" containerID="cri-o://e4d91c4bc51dbb4a0027740e624f6d8a205ec31b987855145ffbd959697fe906" gracePeriod=30
Sep 16 19:20:29.741055 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:29.741021    2578 patch_prober.go:28] interesting pod/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 19:20:29.741228 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:29.741078    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql" podUID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:20:34.833666 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:34.833634    2578 generic.go:356] "Generic (PLEG): container finished" podID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" containerID="e4d91c4bc51dbb4a0027740e624f6d8a205ec31b987855145ffbd959697fe906" exitCode=0
Sep 16 19:20:34.834030 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:34.833723    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql" event={"ID":"73c1758a-a532-4c93-9e2f-c2236f5d58b3","Type":"ContainerDied","Data":"e4d91c4bc51dbb4a0027740e624f6d8a205ec31b987855145ffbd959697fe906"}
Sep 16 19:20:34.834030 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:34.833807    2578 scope.go:117] "RemoveContainer" containerID="86943eaf572e680baa52987808a6ada3527fdfef92875660333c80e6507366c8"
Sep 16 19:20:34.988280 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:34.988206    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:20:35.033222 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033187    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-workload-certs\") pod \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") "
Sep 16 19:20:35.033401 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033233    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vb4wk\" (UniqueName: \"kubernetes.io/projected/73c1758a-a532-4c93-9e2f-c2236f5d58b3-kube-api-access-vb4wk\") pod \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") "
Sep 16 19:20:35.033401 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033288    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-envoy\") pod \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") "
Sep 16 19:20:35.033401 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033308    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-token\") pod \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") "
Sep 16 19:20:35.033401 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033335    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istiod-ca-cert\") pod \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") "
Sep 16 19:20:35.033645 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033428    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-workload-socket\") pod \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") "
Sep 16 19:20:35.033645 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033463    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-data\") pod \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") "
Sep 16 19:20:35.033645 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033502    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-credential-socket\") pod \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") "
Sep 16 19:20:35.033645 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033526    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-podinfo\") pod \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\" (UID: \"73c1758a-a532-4c93-9e2f-c2236f5d58b3\") "
Sep 16 19:20:35.033645 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033538    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "73c1758a-a532-4c93-9e2f-c2236f5d58b3" (UID: "73c1758a-a532-4c93-9e2f-c2236f5d58b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:20:35.033903 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033677    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-data" (OuterVolumeSpecName: "istio-data") pod "73c1758a-a532-4c93-9e2f-c2236f5d58b3" (UID: "73c1758a-a532-4c93-9e2f-c2236f5d58b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:20:35.033903 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033705    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "73c1758a-a532-4c93-9e2f-c2236f5d58b3" (UID: "73c1758a-a532-4c93-9e2f-c2236f5d58b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:20:35.033903 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033756    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "73c1758a-a532-4c93-9e2f-c2236f5d58b3" (UID: "73c1758a-a532-4c93-9e2f-c2236f5d58b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:20:35.033903 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033797    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "73c1758a-a532-4c93-9e2f-c2236f5d58b3" (UID: "73c1758a-a532-4c93-9e2f-c2236f5d58b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:20:35.034130 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033935    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istiod-ca-cert\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:35.034130 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033955    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-workload-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:35.034130 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033969    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-data\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:35.034130 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033983    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-credential-socket\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:35.034130 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.033998    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-workload-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:35.035634 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.035606    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73c1758a-a532-4c93-9e2f-c2236f5d58b3-kube-api-access-vb4wk" (OuterVolumeSpecName: "kube-api-access-vb4wk") pod "73c1758a-a532-4c93-9e2f-c2236f5d58b3" (UID: "73c1758a-a532-4c93-9e2f-c2236f5d58b3"). InnerVolumeSpecName "kube-api-access-vb4wk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:20:35.035865 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.035840    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-token" (OuterVolumeSpecName: "istio-token") pod "73c1758a-a532-4c93-9e2f-c2236f5d58b3" (UID: "73c1758a-a532-4c93-9e2f-c2236f5d58b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:20:35.035950 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.035859    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "73c1758a-a532-4c93-9e2f-c2236f5d58b3" (UID: "73c1758a-a532-4c93-9e2f-c2236f5d58b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:20:35.036145 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.036126    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "73c1758a-a532-4c93-9e2f-c2236f5d58b3" (UID: "73c1758a-a532-4c93-9e2f-c2236f5d58b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:20:35.134650 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.134607    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-envoy\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:35.134650 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.134642    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-token\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:35.134650 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.134654    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73c1758a-a532-4c93-9e2f-c2236f5d58b3-istio-podinfo\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:35.134889 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.134666    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vb4wk\" (UniqueName: \"kubernetes.io/projected/73c1758a-a532-4c93-9e2f-c2236f5d58b3-kube-api-access-vb4wk\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:20:35.841674 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.841641    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql" event={"ID":"73c1758a-a532-4c93-9e2f-c2236f5d58b3","Type":"ContainerDied","Data":"f209ba774e0c1522eecbbce1684a8bb0af71182af9573a8de5d23f4f5ef5170d"}
Sep 16 19:20:35.841674 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.841671    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"
Sep 16 19:20:35.842211 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.841705    2578 scope.go:117] "RemoveContainer" containerID="e4d91c4bc51dbb4a0027740e624f6d8a205ec31b987855145ffbd959697fe906"
Sep 16 19:20:35.866150 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.866117    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"]
Sep 16 19:20:35.871182 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:35.871144    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2dc9fc-openshift-default-74674c969b-xrdql"]
Sep 16 19:20:36.709232 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:20:36.709201    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" path="/var/lib/kubelet/pods/73c1758a-a532-4c93-9e2f-c2236f5d58b3/volumes"
Sep 16 19:23:09.782583 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:09.782548    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"]
Sep 16 19:23:09.783071 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:09.782772    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr" podUID="8098ba1b-391e-4688-98a2-f7ad0a7a72d8" containerName="manager" containerID="cri-o://3edf775a8cd20833d5bb1b45528aa86070f027ab05fbaa961eb1b0b0c0086666" gracePeriod=10
Sep 16 19:23:10.042866 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.042840    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"
Sep 16 19:23:10.107029 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.107000    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rv6f7\" (UniqueName: \"kubernetes.io/projected/8098ba1b-391e-4688-98a2-f7ad0a7a72d8-kube-api-access-rv6f7\") pod \"8098ba1b-391e-4688-98a2-f7ad0a7a72d8\" (UID: \"8098ba1b-391e-4688-98a2-f7ad0a7a72d8\") "
Sep 16 19:23:10.107215 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.107076    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8098ba1b-391e-4688-98a2-f7ad0a7a72d8-extensions-socket-volume\") pod \"8098ba1b-391e-4688-98a2-f7ad0a7a72d8\" (UID: \"8098ba1b-391e-4688-98a2-f7ad0a7a72d8\") "
Sep 16 19:23:10.107568 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.107540    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8098ba1b-391e-4688-98a2-f7ad0a7a72d8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8098ba1b-391e-4688-98a2-f7ad0a7a72d8" (UID: "8098ba1b-391e-4688-98a2-f7ad0a7a72d8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:23:10.109198 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.109144    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8098ba1b-391e-4688-98a2-f7ad0a7a72d8-kube-api-access-rv6f7" (OuterVolumeSpecName: "kube-api-access-rv6f7") pod "8098ba1b-391e-4688-98a2-f7ad0a7a72d8" (UID: "8098ba1b-391e-4688-98a2-f7ad0a7a72d8"). InnerVolumeSpecName "kube-api-access-rv6f7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:23:10.208279 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.208238    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rv6f7\" (UniqueName: \"kubernetes.io/projected/8098ba1b-391e-4688-98a2-f7ad0a7a72d8-kube-api-access-rv6f7\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:23:10.208279 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.208271    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8098ba1b-391e-4688-98a2-f7ad0a7a72d8-extensions-socket-volume\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:23:10.779103 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.779071    2578 generic.go:356] "Generic (PLEG): container finished" podID="8098ba1b-391e-4688-98a2-f7ad0a7a72d8" containerID="3edf775a8cd20833d5bb1b45528aa86070f027ab05fbaa961eb1b0b0c0086666" exitCode=0
Sep 16 19:23:10.779320 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.779125    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"
Sep 16 19:23:10.779320 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.779191    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr" event={"ID":"8098ba1b-391e-4688-98a2-f7ad0a7a72d8","Type":"ContainerDied","Data":"3edf775a8cd20833d5bb1b45528aa86070f027ab05fbaa961eb1b0b0c0086666"}
Sep 16 19:23:10.779320 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.779284    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr" event={"ID":"8098ba1b-391e-4688-98a2-f7ad0a7a72d8","Type":"ContainerDied","Data":"20310e42fe99dbfe7942d030795fe0d7dfcdd3695168979e2df591dd138b2a5b"}
Sep 16 19:23:10.779320 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.779316    2578 scope.go:117] "RemoveContainer" containerID="3edf775a8cd20833d5bb1b45528aa86070f027ab05fbaa961eb1b0b0c0086666"
Sep 16 19:23:10.789112 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.788902    2578 scope.go:117] "RemoveContainer" containerID="1e0ec0656d44329d55c323579124b2750e5e3eddf75bdb991892470caa5cf049"
Sep 16 19:23:10.797309 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.797290    2578 scope.go:117] "RemoveContainer" containerID="3edf775a8cd20833d5bb1b45528aa86070f027ab05fbaa961eb1b0b0c0086666"
Sep 16 19:23:10.797601 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:23:10.797581    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3edf775a8cd20833d5bb1b45528aa86070f027ab05fbaa961eb1b0b0c0086666\": container with ID starting with 3edf775a8cd20833d5bb1b45528aa86070f027ab05fbaa961eb1b0b0c0086666 not found: ID does not exist" containerID="3edf775a8cd20833d5bb1b45528aa86070f027ab05fbaa961eb1b0b0c0086666"
Sep 16 19:23:10.797668 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.797613    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3edf775a8cd20833d5bb1b45528aa86070f027ab05fbaa961eb1b0b0c0086666"} err="failed to get container status \"3edf775a8cd20833d5bb1b45528aa86070f027ab05fbaa961eb1b0b0c0086666\": rpc error: code = NotFound desc = could not find container \"3edf775a8cd20833d5bb1b45528aa86070f027ab05fbaa961eb1b0b0c0086666\": container with ID starting with 3edf775a8cd20833d5bb1b45528aa86070f027ab05fbaa961eb1b0b0c0086666 not found: ID does not exist"
Sep 16 19:23:10.797668 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.797639    2578 scope.go:117] "RemoveContainer" containerID="1e0ec0656d44329d55c323579124b2750e5e3eddf75bdb991892470caa5cf049"
Sep 16 19:23:10.797920 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:23:10.797903    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e0ec0656d44329d55c323579124b2750e5e3eddf75bdb991892470caa5cf049\": container with ID starting with 1e0ec0656d44329d55c323579124b2750e5e3eddf75bdb991892470caa5cf049 not found: ID does not exist" containerID="1e0ec0656d44329d55c323579124b2750e5e3eddf75bdb991892470caa5cf049"
Sep 16 19:23:10.797963 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.797926    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e0ec0656d44329d55c323579124b2750e5e3eddf75bdb991892470caa5cf049"} err="failed to get container status \"1e0ec0656d44329d55c323579124b2750e5e3eddf75bdb991892470caa5cf049\": rpc error: code = NotFound desc = could not find container \"1e0ec0656d44329d55c323579124b2750e5e3eddf75bdb991892470caa5cf049\": container with ID starting with 1e0ec0656d44329d55c323579124b2750e5e3eddf75bdb991892470caa5cf049 not found: ID does not exist"
Sep 16 19:23:10.802568 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.802543    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"]
Sep 16 19:23:10.805690 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:10.805671    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8msmr"]
Sep 16 19:23:12.708967 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:23:12.708933    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8098ba1b-391e-4688-98a2-f7ad0a7a72d8" path="/var/lib/kubelet/pods/8098ba1b-391e-4688-98a2-f7ad0a7a72d8/volumes"
Sep 16 19:24:10.544438 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.544401    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp5lp"]
Sep 16 19:24:10.544984 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.544962    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" containerName="istio-proxy"
Sep 16 19:24:10.545060 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.544988    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" containerName="istio-proxy"
Sep 16 19:24:10.545060 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545001    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" containerName="istio-proxy"
Sep 16 19:24:10.545060 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545009    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" containerName="istio-proxy"
Sep 16 19:24:10.545060 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545018    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4434810-ce86-415a-a913-fc508a548cfb" containerName="istio-proxy"
Sep 16 19:24:10.545060 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545028    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4434810-ce86-415a-a913-fc508a548cfb" containerName="istio-proxy"
Sep 16 19:24:10.545060 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545048    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="818defd5-39e1-44fc-8eaa-a992f44fc6ad" containerName="revoke-keys"
Sep 16 19:24:10.545060 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545057    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="818defd5-39e1-44fc-8eaa-a992f44fc6ad" containerName="revoke-keys"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545070    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a223baca-28f9-49d2-bd57-798b4350d102" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545078    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a223baca-28f9-49d2-bd57-798b4350d102" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545088    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545097    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545109    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545118    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545143    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac10eb3a-a871-4a69-aecd-02c2185b56be" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545152    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac10eb3a-a871-4a69-aecd-02c2185b56be" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545165    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac10eb3a-a871-4a69-aecd-02c2185b56be" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545191    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac10eb3a-a871-4a69-aecd-02c2185b56be" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545202    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8098ba1b-391e-4688-98a2-f7ad0a7a72d8" containerName="manager"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545210    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8098ba1b-391e-4688-98a2-f7ad0a7a72d8" containerName="manager"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545306    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac10eb3a-a871-4a69-aecd-02c2185b56be" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545318    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545329    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8098ba1b-391e-4688-98a2-f7ad0a7a72d8" containerName="manager"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545337    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8098ba1b-391e-4688-98a2-f7ad0a7a72d8" containerName="manager"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545346    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a223baca-28f9-49d2-bd57-798b4350d102" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545355    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545374    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4434810-ce86-415a-a913-fc508a548cfb" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545384    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="53c0234b-6dab-4ce2-bafb-60a4fcebb1bc" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545394    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="818defd5-39e1-44fc-8eaa-a992f44fc6ad" containerName="revoke-keys"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545403    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4434810-ce86-415a-a913-fc508a548cfb" containerName="istio-proxy"
Sep 16 19:24:10.545448 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.545412    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a223baca-28f9-49d2-bd57-798b4350d102" containerName="istio-proxy"
Sep 16 19:24:10.548865 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.548845    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp5lp"
Sep 16 19:24:10.551354 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.551334    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8pkfh\""
Sep 16 19:24:10.565638 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.565612    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5cf8389-f1eb-4186-a59f-4439918b43a5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tp5lp\" (UID: \"b5cf8389-f1eb-4186-a59f-4439918b43a5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp5lp"
Sep 16 19:24:10.565738 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.565668    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjcjd\" (UniqueName: \"kubernetes.io/projected/b5cf8389-f1eb-4186-a59f-4439918b43a5-kube-api-access-pjcjd\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tp5lp\" (UID: \"b5cf8389-f1eb-4186-a59f-4439918b43a5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp5lp"
Sep 16 19:24:10.573585 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.573563    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp5lp"]
Sep 16 19:24:10.666725 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.666681    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5cf8389-f1eb-4186-a59f-4439918b43a5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tp5lp\" (UID: \"b5cf8389-f1eb-4186-a59f-4439918b43a5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp5lp"
Sep 16 19:24:10.666927 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.666772    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjcjd\" (UniqueName: \"kubernetes.io/projected/b5cf8389-f1eb-4186-a59f-4439918b43a5-kube-api-access-pjcjd\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tp5lp\" (UID: \"b5cf8389-f1eb-4186-a59f-4439918b43a5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp5lp"
Sep 16 19:24:10.667116 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.667096    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b5cf8389-f1eb-4186-a59f-4439918b43a5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tp5lp\" (UID: \"b5cf8389-f1eb-4186-a59f-4439918b43a5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp5lp"
Sep 16 19:24:10.678411 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.678377    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjcjd\" (UniqueName: \"kubernetes.io/projected/b5cf8389-f1eb-4186-a59f-4439918b43a5-kube-api-access-pjcjd\") pod \"kuadrant-operator-controller-manager-5769f99d9f-tp5lp\" (UID: \"b5cf8389-f1eb-4186-a59f-4439918b43a5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp5lp"
Sep 16 19:24:10.859100 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:10.859065    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp5lp"
Sep 16 19:24:11.159326 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:11.159298    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp5lp"]
Sep 16 19:24:11.161245 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:24:11.161210    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5cf8389_f1eb_4186_a59f_4439918b43a5.slice/crio-3e1d1b7f97e81351bbe821f5361aeb0a12d44af5c38e28784eee5db1fd63ba6d WatchSource:0}: Error finding container 3e1d1b7f97e81351bbe821f5361aeb0a12d44af5c38e28784eee5db1fd63ba6d: Status 404 returned error can't find the container with id 3e1d1b7f97e81351bbe821f5361aeb0a12d44af5c38e28784eee5db1fd63ba6d
Sep 16 19:24:12.153287 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:12.153242    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp5lp" event={"ID":"b5cf8389-f1eb-4186-a59f-4439918b43a5","Type":"ContainerStarted","Data":"7cb4fc4028d5286c020ca42b2a8df56c9f1ae23d789dd078276d6a2b6904c6b0"}
Sep 16 19:24:12.153902 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:12.153325    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp5lp"
Sep 16 19:24:12.153902 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:12.153351    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp5lp" event={"ID":"b5cf8389-f1eb-4186-a59f-4439918b43a5","Type":"ContainerStarted","Data":"3e1d1b7f97e81351bbe821f5361aeb0a12d44af5c38e28784eee5db1fd63ba6d"}
Sep 16 19:24:12.188597 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:12.188342    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp5lp" podStartSLOduration=2.188321839 podStartE2EDuration="2.188321839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:24:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:24:12.186804696 +0000 UTC m=+2679.974368863" watchObservedRunningTime="2026-09-16 19:24:12.188321839 +0000 UTC m=+2679.975885995"
Sep 16 19:24:23.161609 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:23.161577    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-tp5lp"
Sep 16 19:24:32.997532 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:32.997501    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 19:24:33.003995 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:33.003970    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 19:24:33.005426 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:33.005407    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 19:24:33.011568 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:33.011548    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gbxkd_5bfaaaff-fed0-456e-a757-8655bc43f26d/ovn-acl-logging/0.log"
Sep 16 19:24:33.391599 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:24:33.391564    2578 scope.go:117] "RemoveContainer" containerID="00768b5f921a9a1513e718deb407c6ed1880cefe7789fd34cc28ca361f889ebc"
Sep 16 19:25:51.204320 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.204282    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"]
Sep 16 19:25:51.204797 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.204671    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8098ba1b-391e-4688-98a2-f7ad0a7a72d8" containerName="manager"
Sep 16 19:25:51.204797 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.204682    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8098ba1b-391e-4688-98a2-f7ad0a7a72d8" containerName="manager"
Sep 16 19:25:51.204797 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.204693    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4434810-ce86-415a-a913-fc508a548cfb" containerName="istio-proxy"
Sep 16 19:25:51.204797 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.204699    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4434810-ce86-415a-a913-fc508a548cfb" containerName="istio-proxy"
Sep 16 19:25:51.204797 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.204711    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a223baca-28f9-49d2-bd57-798b4350d102" containerName="istio-proxy"
Sep 16 19:25:51.204797 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.204716    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a223baca-28f9-49d2-bd57-798b4350d102" containerName="istio-proxy"
Sep 16 19:25:51.204797 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.204785    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="73c1758a-a532-4c93-9e2f-c2236f5d58b3" containerName="istio-proxy"
Sep 16 19:25:51.204797 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.204797    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac10eb3a-a871-4a69-aecd-02c2185b56be" containerName="istio-proxy"
Sep 16 19:25:51.207857 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.207836    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:51.211339 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.211305    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 19:25:51.211457 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.211372    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6cp49\""
Sep 16 19:25:51.211457 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.211373    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 16 19:25:51.218973 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.218951    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"]
Sep 16 19:25:51.329862 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.329824    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frm4q\" (UniqueName: \"kubernetes.io/projected/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-kube-api-access-frm4q\") pod \"maas-api-65b6d4d5dc-7wv8c\" (UID: \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\") " pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:51.330037 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.329882    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-metrics-certs\") pod \"maas-api-65b6d4d5dc-7wv8c\" (UID: \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\") " pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:51.330037 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.329969    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-openshift-service-ca\") pod \"maas-api-65b6d4d5dc-7wv8c\" (UID: \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\") " pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:51.330037 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.330009    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-maas-api-tls\") pod \"maas-api-65b6d4d5dc-7wv8c\" (UID: \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\") " pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:51.430774 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.430738    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-maas-api-tls\") pod \"maas-api-65b6d4d5dc-7wv8c\" (UID: \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\") " pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:51.430958 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.430804    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frm4q\" (UniqueName: \"kubernetes.io/projected/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-kube-api-access-frm4q\") pod \"maas-api-65b6d4d5dc-7wv8c\" (UID: \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\") " pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:51.430958 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.430850    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-metrics-certs\") pod \"maas-api-65b6d4d5dc-7wv8c\" (UID: \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\") " pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:51.430958 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.430889    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-openshift-service-ca\") pod \"maas-api-65b6d4d5dc-7wv8c\" (UID: \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\") " pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:51.431632 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.431602    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-openshift-service-ca\") pod \"maas-api-65b6d4d5dc-7wv8c\" (UID: \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\") " pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:51.433317 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.433296    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-metrics-certs\") pod \"maas-api-65b6d4d5dc-7wv8c\" (UID: \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\") " pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:51.433440 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.433361    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-maas-api-tls\") pod \"maas-api-65b6d4d5dc-7wv8c\" (UID: \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\") " pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:51.440753 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.440726    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frm4q\" (UniqueName: \"kubernetes.io/projected/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-kube-api-access-frm4q\") pod \"maas-api-65b6d4d5dc-7wv8c\" (UID: \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\") " pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:51.519619 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.519529    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:51.803471 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.803444    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"]
Sep 16 19:25:51.805963 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:25:51.805929    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0be9ba24_ba24_4a4d_9870_d4c8d91c62c5.slice/crio-fa374bb6efdda27bd1cfd6311153f3305e1f3ae756ea9c1f03ab21b91616de48 WatchSource:0}: Error finding container fa374bb6efdda27bd1cfd6311153f3305e1f3ae756ea9c1f03ab21b91616de48: Status 404 returned error can't find the container with id fa374bb6efdda27bd1cfd6311153f3305e1f3ae756ea9c1f03ab21b91616de48
Sep 16 19:25:51.807455 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:51.807437    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 19:25:52.741887 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:52.741845    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c" event={"ID":"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5","Type":"ContainerStarted","Data":"fa374bb6efdda27bd1cfd6311153f3305e1f3ae756ea9c1f03ab21b91616de48"}
Sep 16 19:25:54.757402 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:54.757366    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c" event={"ID":"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5","Type":"ContainerStarted","Data":"2d1d8c61458454fc07e0ebfb379e753b5c25a8231a776d5018519c3bdc58edcc"}
Sep 16 19:25:54.757742 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:54.757510    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:54.775216 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:54.775144    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c" podStartSLOduration=0.895721963 podStartE2EDuration="3.775128487s" totalImagesPullingTime="2.879406524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:25:54.774157264 +0000 UTC m=+2782.561721443" watchObservedRunningTime="2026-09-16 19:25:54.775128487 +0000 UTC m=+2782.562692641"
Sep 16 19:25:58.884011 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:58.883976    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"]
Sep 16 19:25:58.884446 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:58.884217    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c" podUID="0be9ba24-ba24-4a4d-9870-d4c8d91c62c5" containerName="maas-api" containerID="cri-o://2d1d8c61458454fc07e0ebfb379e753b5c25a8231a776d5018519c3bdc58edcc" gracePeriod=30
Sep 16 19:25:58.920881 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:58.920847    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-756969d76c-fr825"]
Sep 16 19:25:58.924541 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:58.924524    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825"
Sep 16 19:25:58.936080 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:58.936049    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-756969d76c-fr825"]
Sep 16 19:25:59.004980 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.004940    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbkmr\" (UniqueName: \"kubernetes.io/projected/ab7d03cc-a760-4692-9483-c62cfcd5b9bb-kube-api-access-dbkmr\") pod \"maas-api-756969d76c-fr825\" (UID: \"ab7d03cc-a760-4692-9483-c62cfcd5b9bb\") " pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825"
Sep 16 19:25:59.004980 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.004983    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab7d03cc-a760-4692-9483-c62cfcd5b9bb-openshift-service-ca\") pod \"maas-api-756969d76c-fr825\" (UID: \"ab7d03cc-a760-4692-9483-c62cfcd5b9bb\") " pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825"
Sep 16 19:25:59.005243 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.005000    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab7d03cc-a760-4692-9483-c62cfcd5b9bb-metrics-certs\") pod \"maas-api-756969d76c-fr825\" (UID: \"ab7d03cc-a760-4692-9483-c62cfcd5b9bb\") " pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825"
Sep 16 19:25:59.005243 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.005161    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab7d03cc-a760-4692-9483-c62cfcd5b9bb-maas-api-tls\") pod \"maas-api-756969d76c-fr825\" (UID: \"ab7d03cc-a760-4692-9483-c62cfcd5b9bb\") " pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825"
Sep 16 19:25:59.105888 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.105850    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbkmr\" (UniqueName: \"kubernetes.io/projected/ab7d03cc-a760-4692-9483-c62cfcd5b9bb-kube-api-access-dbkmr\") pod \"maas-api-756969d76c-fr825\" (UID: \"ab7d03cc-a760-4692-9483-c62cfcd5b9bb\") " pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825"
Sep 16 19:25:59.106091 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.105905    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab7d03cc-a760-4692-9483-c62cfcd5b9bb-openshift-service-ca\") pod \"maas-api-756969d76c-fr825\" (UID: \"ab7d03cc-a760-4692-9483-c62cfcd5b9bb\") " pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825"
Sep 16 19:25:59.106091 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.105928    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab7d03cc-a760-4692-9483-c62cfcd5b9bb-metrics-certs\") pod \"maas-api-756969d76c-fr825\" (UID: \"ab7d03cc-a760-4692-9483-c62cfcd5b9bb\") " pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825"
Sep 16 19:25:59.106091 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.106020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab7d03cc-a760-4692-9483-c62cfcd5b9bb-maas-api-tls\") pod \"maas-api-756969d76c-fr825\" (UID: \"ab7d03cc-a760-4692-9483-c62cfcd5b9bb\") " pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825"
Sep 16 19:25:59.106733 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.106710    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab7d03cc-a760-4692-9483-c62cfcd5b9bb-openshift-service-ca\") pod \"maas-api-756969d76c-fr825\" (UID: \"ab7d03cc-a760-4692-9483-c62cfcd5b9bb\") " pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825"
Sep 16 19:25:59.108641 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.108615    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab7d03cc-a760-4692-9483-c62cfcd5b9bb-metrics-certs\") pod \"maas-api-756969d76c-fr825\" (UID: \"ab7d03cc-a760-4692-9483-c62cfcd5b9bb\") " pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825"
Sep 16 19:25:59.108720 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.108695    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab7d03cc-a760-4692-9483-c62cfcd5b9bb-maas-api-tls\") pod \"maas-api-756969d76c-fr825\" (UID: \"ab7d03cc-a760-4692-9483-c62cfcd5b9bb\") " pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825"
Sep 16 19:25:59.119156 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.119134    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbkmr\" (UniqueName: \"kubernetes.io/projected/ab7d03cc-a760-4692-9483-c62cfcd5b9bb-kube-api-access-dbkmr\") pod \"maas-api-756969d76c-fr825\" (UID: \"ab7d03cc-a760-4692-9483-c62cfcd5b9bb\") " pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825"
Sep 16 19:25:59.140034 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.139966    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:59.257547 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.257513    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825"
Sep 16 19:25:59.308423 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.308393    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-metrics-certs\") pod \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\" (UID: \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\") "
Sep 16 19:25:59.308594 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.308490    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-maas-api-tls\") pod \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\" (UID: \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\") "
Sep 16 19:25:59.308594 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.308526    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-openshift-service-ca\") pod \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\" (UID: \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\") "
Sep 16 19:25:59.308594 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.308557    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frm4q\" (UniqueName: \"kubernetes.io/projected/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-kube-api-access-frm4q\") pod \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\" (UID: \"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5\") "
Sep 16 19:25:59.309301 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.309267    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0be9ba24-ba24-4a4d-9870-d4c8d91c62c5" (UID: "0be9ba24-ba24-4a4d-9870-d4c8d91c62c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:25:59.311858 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.311812    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-kube-api-access-frm4q" (OuterVolumeSpecName: "kube-api-access-frm4q") pod "0be9ba24-ba24-4a4d-9870-d4c8d91c62c5" (UID: "0be9ba24-ba24-4a4d-9870-d4c8d91c62c5"). InnerVolumeSpecName "kube-api-access-frm4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:25:59.311958 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.311916    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0be9ba24-ba24-4a4d-9870-d4c8d91c62c5" (UID: "0be9ba24-ba24-4a4d-9870-d4c8d91c62c5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:25:59.312049 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.311993    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0be9ba24-ba24-4a4d-9870-d4c8d91c62c5" (UID: "0be9ba24-ba24-4a4d-9870-d4c8d91c62c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:25:59.410105 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.410005    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-metrics-certs\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:25:59.410105 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.410047    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-maas-api-tls\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:25:59.410105 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.410064    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-openshift-service-ca\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:25:59.410105 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.410079    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frm4q\" (UniqueName: \"kubernetes.io/projected/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5-kube-api-access-frm4q\") on node \"ip-10-0-134-234.ec2.internal\" DevicePath \"\""
Sep 16 19:25:59.537059 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.537033    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-756969d76c-fr825"]
Sep 16 19:25:59.538610 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:25:59.538571    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab7d03cc_a760_4692_9483_c62cfcd5b9bb.slice/crio-b4732ec9b3317087c4ac9bca763c2a37c0c186f6a79f6070f6a684c7ba1cbbdb WatchSource:0}: Error finding container b4732ec9b3317087c4ac9bca763c2a37c0c186f6a79f6070f6a684c7ba1cbbdb: Status 404 returned error can't find the container with id b4732ec9b3317087c4ac9bca763c2a37c0c186f6a79f6070f6a684c7ba1cbbdb
Sep 16 19:25:59.791018 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.790922    2578 generic.go:356] "Generic (PLEG): container finished" podID="0be9ba24-ba24-4a4d-9870-d4c8d91c62c5" containerID="2d1d8c61458454fc07e0ebfb379e753b5c25a8231a776d5018519c3bdc58edcc" exitCode=0
Sep 16 19:25:59.791018 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.790994    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"
Sep 16 19:25:59.791251 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.791011    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c" event={"ID":"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5","Type":"ContainerDied","Data":"2d1d8c61458454fc07e0ebfb379e753b5c25a8231a776d5018519c3bdc58edcc"}
Sep 16 19:25:59.791251 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.791090    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c" event={"ID":"0be9ba24-ba24-4a4d-9870-d4c8d91c62c5","Type":"ContainerDied","Data":"fa374bb6efdda27bd1cfd6311153f3305e1f3ae756ea9c1f03ab21b91616de48"}
Sep 16 19:25:59.791251 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.791127    2578 scope.go:117] "RemoveContainer" containerID="2d1d8c61458454fc07e0ebfb379e753b5c25a8231a776d5018519c3bdc58edcc"
Sep 16 19:25:59.793325 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.793297    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825" event={"ID":"ab7d03cc-a760-4692-9483-c62cfcd5b9bb","Type":"ContainerStarted","Data":"b4732ec9b3317087c4ac9bca763c2a37c0c186f6a79f6070f6a684c7ba1cbbdb"}
Sep 16 19:25:59.802650 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.802571    2578 scope.go:117] "RemoveContainer" containerID="2d1d8c61458454fc07e0ebfb379e753b5c25a8231a776d5018519c3bdc58edcc"
Sep 16 19:25:59.803400 ip-10-0-134-234 kubenswrapper[2578]: E0916 19:25:59.803367    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d1d8c61458454fc07e0ebfb379e753b5c25a8231a776d5018519c3bdc58edcc\": container with ID starting with 2d1d8c61458454fc07e0ebfb379e753b5c25a8231a776d5018519c3bdc58edcc not found: ID does not exist" containerID="2d1d8c61458454fc07e0ebfb379e753b5c25a8231a776d5018519c3bdc58edcc"
Sep 16 19:25:59.803517 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.803411    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d1d8c61458454fc07e0ebfb379e753b5c25a8231a776d5018519c3bdc58edcc"} err="failed to get container status \"2d1d8c61458454fc07e0ebfb379e753b5c25a8231a776d5018519c3bdc58edcc\": rpc error: code = NotFound desc = could not find container \"2d1d8c61458454fc07e0ebfb379e753b5c25a8231a776d5018519c3bdc58edcc\": container with ID starting with 2d1d8c61458454fc07e0ebfb379e753b5c25a8231a776d5018519c3bdc58edcc not found: ID does not exist"
Sep 16 19:25:59.820306 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.820279    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"]
Sep 16 19:25:59.824403 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:25:59.824373    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-65b6d4d5dc-7wv8c"]
Sep 16 19:26:00.710802 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:00.710770    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0be9ba24-ba24-4a4d-9870-d4c8d91c62c5" path="/var/lib/kubelet/pods/0be9ba24-ba24-4a4d-9870-d4c8d91c62c5/volumes"
Sep 16 19:26:00.802136 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:00.802097    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825" event={"ID":"ab7d03cc-a760-4692-9483-c62cfcd5b9bb","Type":"ContainerStarted","Data":"b9d2e02386281511c2581858a25b0731336e64918f8282595f341934a9c5e732"}
Sep 16 19:26:00.802358 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:00.802217    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825"
Sep 16 19:26:00.827795 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:00.827743    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825" podStartSLOduration=2.32814829 podStartE2EDuration="2.827725183s" totalImagesPullingTime="499.576893ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:26:00.825358911 +0000 UTC m=+2788.612923082" watchObservedRunningTime="2026-09-16 19:26:00.827725183 +0000 UTC m=+2788.615289337"
Sep 16 19:26:06.814428 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:06.814400    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-756969d76c-fr825"
Sep 16 19:26:26.889780 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:26.889748    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85745479c7-5jz79_0a308b28-fbda-4df4-b85c-ec0cf9b956bc/manager/0.log"
Sep 16 19:26:26.897074 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:26.897050    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85745479c7-5jz79_0a308b28-fbda-4df4-b85c-ec0cf9b956bc/copy-manifests/0.log"
Sep 16 19:26:27.478765 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:27.478737    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-756969d76c-fr825_ab7d03cc-a760-4692-9483-c62cfcd5b9bb/maas-api/0.log"
Sep 16 19:26:27.690071 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:27.690040    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29826420-w47t2_dbc9ab59-350e-408a-82f1-b6e59404c51f/cleanup/0.log"
Sep 16 19:26:27.798635 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:27.798546    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29826435-gfkxv_976e845e-b2fc-436b-9c41-60b652bb5354/cleanup/0.log"
Sep 16 19:26:27.904947 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:27.904919    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-sz4pq_d3908004-d532-480f-a0f4-331659a9a00a/postgres/0.log"
Sep 16 19:26:28.780050 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:28.780005    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-tfkdv_7e508345-cf51-4288-a312-79df0e23fcbf/manager/0.log"
Sep 16 19:26:29.103109 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:29.103031    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-tp5lp_b5cf8389-f1eb-4186-a59f-4439918b43a5/manager/0.log"
Sep 16 19:26:29.676810 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:29.676777    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl_f181395c-866f-4dd2-90d4-c316e2d0bf39/istio-proxy/0.log"
Sep 16 19:26:29.792816 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:29.792785    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-gjwmr_ac21262d-9d0c-42cf-ad35-50819dfdeeed/discovery/0.log"
Sep 16 19:26:29.900439 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:29.900409    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d78bff8dd-bsvjf_48194838-f8d0-4b1a-a3b3-0ad28d1076a1/kube-auth-proxy/0.log"
Sep 16 19:26:30.117148 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:30.117120    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rcxf8_85290364-ea9e-4f0f-ae40-fb741b4fbc68/istio-proxy/1.log"
Sep 16 19:26:30.225120 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:30.225088    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hctdl_e9af41fc-a845-4391-ac4c-7e1ee182f84f/payload-pre-processing/0.log"
Sep 16 19:26:30.870658 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:30.870630    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qqjl4_b2045438-5d90-4175-8005-91331a08ac76/main/0.log"
Sep 16 19:26:31.305953 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:31.305872    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-t7jpp_7bdcf741-3ca7-463e-ace8-b261652b59c9/main/0.log"
Sep 16 19:26:40.098709 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:40.098677    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-c8l7h_73a1706e-dce9-475e-b553-642999cc42c0/global-pull-secret-syncer/0.log"
Sep 16 19:26:40.356792 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:40.356768    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sdmdp_ffef2c6f-fe5b-4b5e-9d95-e7eac9b4f8be/konnectivity-agent/0.log"
Sep 16 19:26:40.381940 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:40.381914    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-234.ec2.internal_41ec96924e9815932b677eedb2f521c7/haproxy/0.log"
Sep 16 19:26:44.297425 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:44.297394    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-tfkdv_7e508345-cf51-4288-a312-79df0e23fcbf/manager/0.log"
Sep 16 19:26:44.412794 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:44.412760    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-tp5lp_b5cf8389-f1eb-4186-a59f-4439918b43a5/manager/0.log"
Sep 16 19:26:46.383087 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:46.383052    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-4vqx5_54176761-3f77-4f63-ada8-c65ec07f450c/cluster-monitoring-operator/0.log"
Sep 16 19:26:46.632972 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:46.632938    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-clkzl_df0ff542-aa25-4096-b1d5-3064c5b308b3/node-exporter/0.log"
Sep 16 19:26:46.653848 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:46.653779    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-clkzl_df0ff542-aa25-4096-b1d5-3064c5b308b3/kube-rbac-proxy/0.log"
Sep 16 19:26:46.675406 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:46.675384    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-clkzl_df0ff542-aa25-4096-b1d5-3064c5b308b3/init-textfile/0.log"
Sep 16 19:26:46.783792 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:46.783750    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-7jfq8_b8172967-90c2-4159-9b6d-b0838c6703af/kube-rbac-proxy-main/0.log"
Sep 16 19:26:46.805692 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:46.805661    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-7jfq8_b8172967-90c2-4159-9b6d-b0838c6703af/kube-rbac-proxy-self/0.log"
Sep 16 19:26:46.828487 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:46.828458    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-7jfq8_b8172967-90c2-4159-9b6d-b0838c6703af/openshift-state-metrics/0.log"
Sep 16 19:26:47.082131 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:47.082052    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-l874g_25bcf7ce-8826-435a-8c06-7a229e704bf3/prometheus-operator/0.log"
Sep 16 19:26:47.129591 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:47.129561    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-l874g_25bcf7ce-8826-435a-8c06-7a229e704bf3/kube-rbac-proxy/0.log"
Sep 16 19:26:47.215521 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:47.215489    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c6584dd76-xsv84_12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc/telemeter-client/0.log"
Sep 16 19:26:47.240378 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:47.240349    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c6584dd76-xsv84_12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc/reload/0.log"
Sep 16 19:26:47.267431 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:47.267399    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c6584dd76-xsv84_12dd3f0e-d348-4a70-a2b9-eb9922ee0dcc/kube-rbac-proxy/0.log"
Sep 16 19:26:49.084379 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:49.084348    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-vr97q_6d1b4b46-d618-4ceb-8e69-d034dce9b435/networking-console-plugin/0.log"
Sep 16 19:26:49.757153 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:49.757105    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/2.log"
Sep 16 19:26:49.760320 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:49.760303    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-ttc6f_845d9b0a-5790-4a60-894b-f6b9adce026d/console-operator/3.log"
Sep 16 19:26:50.481197 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:50.481152    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-mthnh_ee22aa60-e093-4d9b-87bb-46d5e6fdd649/download-server/0.log"
Sep 16 19:26:51.042293 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.042259    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"]
Sep 16 19:26:51.042696 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.042683    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0be9ba24-ba24-4a4d-9870-d4c8d91c62c5" containerName="maas-api"
Sep 16 19:26:51.042743 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.042698    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0be9ba24-ba24-4a4d-9870-d4c8d91c62c5" containerName="maas-api"
Sep 16 19:26:51.042795 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.042786    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0be9ba24-ba24-4a4d-9870-d4c8d91c62c5" containerName="maas-api"
Sep 16 19:26:51.044854 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.044836    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.048073 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.048048    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mrp4v\"/\"openshift-service-ca.crt\""
Sep 16 19:26:51.049204 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.049185    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mrp4v\"/\"kube-root-ca.crt\""
Sep 16 19:26:51.049296 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.049235    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mrp4v\"/\"default-dockercfg-nn9pg\""
Sep 16 19:26:51.055743 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.055718    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"]
Sep 16 19:26:51.104182 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.104143    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cdae7814-723d-4fdd-b804-e175767a6818-lib-modules\") pod \"perf-node-gather-daemonset-nj7qm\" (UID: \"cdae7814-723d-4fdd-b804-e175767a6818\") " pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.104363 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.104257    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt2wj\" (UniqueName: \"kubernetes.io/projected/cdae7814-723d-4fdd-b804-e175767a6818-kube-api-access-nt2wj\") pod \"perf-node-gather-daemonset-nj7qm\" (UID: \"cdae7814-723d-4fdd-b804-e175767a6818\") " pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.104363 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.104315    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cdae7814-723d-4fdd-b804-e175767a6818-sys\") pod \"perf-node-gather-daemonset-nj7qm\" (UID: \"cdae7814-723d-4fdd-b804-e175767a6818\") " pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.104363 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.104337    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cdae7814-723d-4fdd-b804-e175767a6818-podres\") pod \"perf-node-gather-daemonset-nj7qm\" (UID: \"cdae7814-723d-4fdd-b804-e175767a6818\") " pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.104468 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.104436    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cdae7814-723d-4fdd-b804-e175767a6818-proc\") pod \"perf-node-gather-daemonset-nj7qm\" (UID: \"cdae7814-723d-4fdd-b804-e175767a6818\") " pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.202967 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.202939    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-b9d6k_6a1e4944-7d0e-4a4c-8cd0-b1a762f4798c/volume-data-source-validator/0.log"
Sep 16 19:26:51.204924 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.204901    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cdae7814-723d-4fdd-b804-e175767a6818-lib-modules\") pod \"perf-node-gather-daemonset-nj7qm\" (UID: \"cdae7814-723d-4fdd-b804-e175767a6818\") " pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.205058 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.204954    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt2wj\" (UniqueName: \"kubernetes.io/projected/cdae7814-723d-4fdd-b804-e175767a6818-kube-api-access-nt2wj\") pod \"perf-node-gather-daemonset-nj7qm\" (UID: \"cdae7814-723d-4fdd-b804-e175767a6818\") " pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.205058 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.205011    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cdae7814-723d-4fdd-b804-e175767a6818-sys\") pod \"perf-node-gather-daemonset-nj7qm\" (UID: \"cdae7814-723d-4fdd-b804-e175767a6818\") " pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.205058 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.205039    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cdae7814-723d-4fdd-b804-e175767a6818-podres\") pod \"perf-node-gather-daemonset-nj7qm\" (UID: \"cdae7814-723d-4fdd-b804-e175767a6818\") " pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.205251 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.205068    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cdae7814-723d-4fdd-b804-e175767a6818-lib-modules\") pod \"perf-node-gather-daemonset-nj7qm\" (UID: \"cdae7814-723d-4fdd-b804-e175767a6818\") " pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.205251 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.205082    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cdae7814-723d-4fdd-b804-e175767a6818-proc\") pod \"perf-node-gather-daemonset-nj7qm\" (UID: \"cdae7814-723d-4fdd-b804-e175767a6818\") " pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.205251 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.205103    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cdae7814-723d-4fdd-b804-e175767a6818-sys\") pod \"perf-node-gather-daemonset-nj7qm\" (UID: \"cdae7814-723d-4fdd-b804-e175767a6818\") " pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.205251 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.205141    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cdae7814-723d-4fdd-b804-e175767a6818-podres\") pod \"perf-node-gather-daemonset-nj7qm\" (UID: \"cdae7814-723d-4fdd-b804-e175767a6818\") " pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.205251 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.205213    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cdae7814-723d-4fdd-b804-e175767a6818-proc\") pod \"perf-node-gather-daemonset-nj7qm\" (UID: \"cdae7814-723d-4fdd-b804-e175767a6818\") " pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.213460 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.213441    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt2wj\" (UniqueName: \"kubernetes.io/projected/cdae7814-723d-4fdd-b804-e175767a6818-kube-api-access-nt2wj\") pod \"perf-node-gather-daemonset-nj7qm\" (UID: \"cdae7814-723d-4fdd-b804-e175767a6818\") " pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.355909 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.355806    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:51.628236 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:51.628210    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"]
Sep 16 19:26:51.628975 ip-10-0-134-234 kubenswrapper[2578]: W0916 19:26:51.628945    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcdae7814_723d_4fdd_b804_e175767a6818.slice/crio-344c5d8d16abcc07ffd3ce592abb5c530cd5af7fe85e4ecaf33a2a137805cff5 WatchSource:0}: Error finding container 344c5d8d16abcc07ffd3ce592abb5c530cd5af7fe85e4ecaf33a2a137805cff5: Status 404 returned error can't find the container with id 344c5d8d16abcc07ffd3ce592abb5c530cd5af7fe85e4ecaf33a2a137805cff5
Sep 16 19:26:52.122851 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:52.122814    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm" event={"ID":"cdae7814-723d-4fdd-b804-e175767a6818","Type":"ContainerStarted","Data":"3382aec41fb1059189b83348e3ca06fd3ffa41111c238e9adafff738808d11de"}
Sep 16 19:26:52.123019 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:52.122975    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:52.123064 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:52.123020    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm" event={"ID":"cdae7814-723d-4fdd-b804-e175767a6818","Type":"ContainerStarted","Data":"344c5d8d16abcc07ffd3ce592abb5c530cd5af7fe85e4ecaf33a2a137805cff5"}
Sep 16 19:26:52.141947 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:52.141896    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm" podStartSLOduration=1.141877078 podStartE2EDuration="1.141877078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 19:26:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:26:52.139095138 +0000 UTC m=+2839.926659310" watchObservedRunningTime="2026-09-16 19:26:52.141877078 +0000 UTC m=+2839.929441236"
Sep 16 19:26:52.356061 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:52.356032    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-crt2q_6fbdb2ba-c71f-47e6-bb39-1f2aa806df61/dns/0.log"
Sep 16 19:26:52.380903 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:52.380826    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-crt2q_6fbdb2ba-c71f-47e6-bb39-1f2aa806df61/kube-rbac-proxy/0.log"
Sep 16 19:26:52.457042 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:52.457010    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-57l9p_d96fa3e0-be40-4f37-99be-57261dce04d1/dns-node-resolver/0.log"
Sep 16 19:26:53.132161 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:53.132129    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-68974dbd59-lgnkg_9cebc0d4-b215-494b-b236-32640184278b/registry/0.log"
Sep 16 19:26:53.184249 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:53.184219    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-478xv_fae250ed-5b59-4388-af13-fcc8d0446e5d/node-ca/0.log"
Sep 16 19:26:54.256926 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:54.256895    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl_f181395c-866f-4dd2-90d4-c316e2d0bf39/istio-proxy/0.log"
Sep 16 19:26:54.706440 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:54.706404    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-gjwmr_ac21262d-9d0c-42cf-ad35-50819dfdeeed/discovery/0.log"
Sep 16 19:26:54.730342 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:54.730314    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d78bff8dd-bsvjf_48194838-f8d0-4b1a-a3b3-0ad28d1076a1/kube-auth-proxy/0.log"
Sep 16 19:26:54.793231 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:54.793194    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rcxf8_85290364-ea9e-4f0f-ae40-fb741b4fbc68/istio-proxy/0.log"
Sep 16 19:26:54.796034 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:54.796012    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rcxf8_85290364-ea9e-4f0f-ae40-fb741b4fbc68/istio-proxy/1.log"
Sep 16 19:26:54.826953 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:54.826922    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hctdl_e9af41fc-a845-4391-ac4c-7e1ee182f84f/payload-pre-processing/0.log"
Sep 16 19:26:55.809778 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:55.809746    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-h6s5l_f21dc980-1bae-414a-be6b-7145bcc7c7f5/serve-healthcheck-canary/0.log"
Sep 16 19:26:56.389908 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:56.389877    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-vhrf8_306e53f3-37de-49c6-a491-f9faf67d7f2a/insights-operator/0.log"
Sep 16 19:26:56.390276 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:56.390258    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-vhrf8_306e53f3-37de-49c6-a491-f9faf67d7f2a/insights-operator/1.log"
Sep 16 19:26:56.550127 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:56.550098    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-brz48_530a3e11-7a4d-4b54-aac7-72ae0e5f3e95/kube-rbac-proxy/0.log"
Sep 16 19:26:56.585194 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:56.585146    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-brz48_530a3e11-7a4d-4b54-aac7-72ae0e5f3e95/exporter/0.log"
Sep 16 19:26:56.615637 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:56.615608    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-brz48_530a3e11-7a4d-4b54-aac7-72ae0e5f3e95/extractor/0.log"
Sep 16 19:26:58.138993 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:58.138959    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mrp4v/perf-node-gather-daemonset-nj7qm"
Sep 16 19:26:59.365769 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:59.365742    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85745479c7-5jz79_0a308b28-fbda-4df4-b85c-ec0cf9b956bc/manager/0.log"
Sep 16 19:26:59.390531 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:26:59.390506    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85745479c7-5jz79_0a308b28-fbda-4df4-b85c-ec0cf9b956bc/copy-manifests/0.log"
Sep 16 19:27:00.618999 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:27:00.618965    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl_f181395c-866f-4dd2-90d4-c316e2d0bf39/istio-proxy/0.log"
Sep 16 19:27:01.023165 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:27:01.023135    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-2qsnr_ad015505-63ef-4a46-acee-ff779e3ce4e2/openshift-lws-operator/0.log"
Sep 16 19:27:01.049112 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:27:01.049085    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-gjwmr_ac21262d-9d0c-42cf-ad35-50819dfdeeed/discovery/0.log"
Sep 16 19:27:01.067136 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:27:01.067108    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d78bff8dd-bsvjf_48194838-f8d0-4b1a-a3b3-0ad28d1076a1/kube-auth-proxy/0.log"
Sep 16 19:27:01.142943 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:27:01.142908    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rcxf8_85290364-ea9e-4f0f-ae40-fb741b4fbc68/istio-proxy/0.log"
Sep 16 19:27:01.152212 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:27:01.152187    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rcxf8_85290364-ea9e-4f0f-ae40-fb741b4fbc68/istio-proxy/1.log"
Sep 16 19:27:01.187200 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:27:01.187159    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hctdl_e9af41fc-a845-4391-ac4c-7e1ee182f84f/payload-pre-processing/0.log"
Sep 16 19:27:02.730748 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:27:02.730720    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cm9jsl_f181395c-866f-4dd2-90d4-c316e2d0bf39/istio-proxy/0.log"
Sep 16 19:27:03.157580 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:27:03.157552    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-gjwmr_ac21262d-9d0c-42cf-ad35-50819dfdeeed/discovery/0.log"
Sep 16 19:27:03.176653 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:27:03.176625    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d78bff8dd-bsvjf_48194838-f8d0-4b1a-a3b3-0ad28d1076a1/kube-auth-proxy/0.log"
Sep 16 19:27:03.220295 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:27:03.220264    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rcxf8_85290364-ea9e-4f0f-ae40-fb741b4fbc68/istio-proxy/0.log"
Sep 16 19:27:03.222960 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:27:03.222940    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rcxf8_85290364-ea9e-4f0f-ae40-fb741b4fbc68/istio-proxy/1.log"
Sep 16 19:27:03.247525 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:27:03.247497    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-hctdl_e9af41fc-a845-4391-ac4c-7e1ee182f84f/payload-pre-processing/0.log"
Sep 16 19:27:08.489163 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:27:08.489129    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nsrzq_1df53334-7b61-47d0-9787-1dc604c84896/kube-multus-additional-cni-plugins/0.log"
Sep 16 19:27:08.512420 ip-10-0-134-234 kubenswrapper[2578]: I0916 19:27:08.512385    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nsrzq_1df53334-7b61-47d0-9787-1dc604c84896/egress-router-binary-copy/0.log"