Sep 08 20:23:29.925132 ip-10-0-136-142 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 08 20:23:29.925147 ip-10-0-136-142 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 08 20:23:29.925158 ip-10-0-136-142 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 08 20:23:29.925489 ip-10-0-136-142 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 08 20:23:39.636916 ip-10-0-136-142 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 08 20:23:39.636932 ip-10-0-136-142 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e9fc02488ee24d17a7061bce69a104ab --
Sep 08 20:25:26.334287 ip-10-0-136-142 systemd[1]: Starting Kubernetes Kubelet...
Sep 08 20:25:26.732070 ip-10-0-136-142 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 20:25:26.732070 ip-10-0-136-142 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 08 20:25:26.732070 ip-10-0-136-142 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 20:25:26.732070 ip-10-0-136-142 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 08 20:25:26.732070 ip-10-0-136-142 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 20:25:26.734032 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.733925    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 08 20:25:26.738244 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738213    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 20:25:26.738244 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738235    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 20:25:26.738244 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738239    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 20:25:26.738244 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738242    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 20:25:26.738244 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738245    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 20:25:26.738244 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738249    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 20:25:26.738244 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738252    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 20:25:26.738244 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738256    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738258    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738261    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738264    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738267    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738270    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738272    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738275    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738278    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738280    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738283    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738285    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738288    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738292    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738296    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738299    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738301    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738304    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738307    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738309    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 20:25:26.738544 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738312    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738315    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738318    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738320    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738323    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738326    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738329    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738331    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738335    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738338    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738340    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738343    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738345    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738348    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738351    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738354    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738357    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738360    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738362    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738365    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 20:25:26.739040 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738368    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738370    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738373    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738376    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738379    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738382    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738386    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738391    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738394    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738397    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738400    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738402    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738405    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738408    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738410    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738413    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738415    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738418    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738421    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 20:25:26.739566 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738425    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738429    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738434    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738437    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738439    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738442    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738445    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738448    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738451    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738454    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738457    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738459    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738462    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738465    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738468    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738471    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738473    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738482    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738485    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738487    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 20:25:26.740050 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738490    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738493    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738495    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738498    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738500    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738503    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738506    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738508    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738511    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738513    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738516    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738518    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.738521    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739302    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739309    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739313    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739317    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739320    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739323    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 20:25:26.740550 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739326    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739329    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739333    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739337    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739340    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739343    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739346    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739349    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739352    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739355    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739357    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739361    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739364    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739367    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739370    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739373    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739376    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739378    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739381    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 20:25:26.741024 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739383    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739386    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739389    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739391    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739394    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739397    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739399    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739403    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739406    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739408    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739412    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739415    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739418    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739421    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739423    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739425    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739428    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739430    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739433    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739435    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 20:25:26.741505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739438    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739441    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739443    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739446    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739448    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739451    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739454    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739456    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739459    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739463    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739466    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739469    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739472    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739474    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739479    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739481    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739484    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739487    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739489    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 20:25:26.742087 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739492    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739495    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739497    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739500    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739503    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739506    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739509    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739511    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739514    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739516    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739519    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739522    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739524    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739526    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739529    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739531    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739534    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739537    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739539    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 20:25:26.742618 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739543    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739545    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739548    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739550    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739553    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739555    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739558    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739560    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739563    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739566    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739569    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739571    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739574    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739577    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739580    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.739583    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740026    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740033    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740036    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740039    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 20:25:26.743097 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740042    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740044    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740047    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740050    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740053    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740055    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740058    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740061    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740064    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740067    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740069    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740072    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740075    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740078    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740080    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740083    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740085    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740088    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740090    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740093    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 20:25:26.743584 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740095    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740098    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740100    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740103    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740105    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740108    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740111    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740113    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740116    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740119    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740121    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740124    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740126    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740129    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740132    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740134    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740137    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740139    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740142    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 20:25:26.744094 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740144    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740148    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740152    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740155    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740157    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740160    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740164    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740166    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740169    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740171    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740174    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740176    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740179    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740181    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740184    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740187    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740190    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740192    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740198    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 20:25:26.744562 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740200    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740203    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740206    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740208    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740211    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740215    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740218    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740221    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740224    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740227    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740229    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740232    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740234    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740237    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740239    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740242    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740244    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740247    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740249    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740254    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 20:25:26.745075 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740259    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740262    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740265    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740268    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740271    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740274    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740277    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740280    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740282    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740285    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740288    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740291    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740294    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740296    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740299    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740301    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.740304    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740384    2580 flags.go:64] FLAG: --address="0.0.0.0"
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740421    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740429    2580 flags.go:64] FLAG: --anonymous-auth="true"
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740434    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 08 20:25:26.745561 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740438    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740442    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740446    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740450    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740454    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740457    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740460    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740464    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740467    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740470    2580 flags.go:64] FLAG: --cgroup-root=""
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740473    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740477    2580 flags.go:64] FLAG: --client-ca-file=""
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740484    2580 flags.go:64] FLAG: --cloud-provider="external"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740487    2580 flags.go:64] FLAG: --cluster-dns="[]"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740491    2580 flags.go:64] FLAG: --cluster-domain=""
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740494    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740497    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740500    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740504    2580 flags.go:64] FLAG: --container-log-max-files="5"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740507    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740510    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740513    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740518    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740521    2580 flags.go:64] FLAG: --contention-profiling="false"
Sep 08 20:25:26.746090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740524    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740527    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740531    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740533    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740538    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740541    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740544    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740546    2580 flags.go:64] FLAG: --enable-load-reader="false"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740549    2580 flags.go:64] FLAG: --enable-server="true"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740552    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740559    2580 flags.go:64] FLAG: --event-burst="100"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740563    2580 flags.go:64] FLAG: --event-qps="50"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740565    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740568    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740571    2580 flags.go:64] FLAG: --eviction-hard=""
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740575    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740578    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740581    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740584    2580 flags.go:64] FLAG: --eviction-soft=""
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740588    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740591    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740594    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740597    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740600    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740603    2580 flags.go:64] FLAG: --fail-swap-on="true"
Sep 08 20:25:26.746676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740605    2580 flags.go:64] FLAG: --feature-gates=""
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740610    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740613    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740616    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740619    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740622    2580 flags.go:64] FLAG: --healthz-port="10248"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740626    2580 flags.go:64] FLAG: --help="false"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740628    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-136-142.ec2.internal"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740631    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740634    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740637    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740640    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740643    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740646    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740649    2580 flags.go:64] FLAG: --image-service-endpoint=""
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740651    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740654    2580 flags.go:64] FLAG: --kube-api-burst="100"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740657    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740660    2580 flags.go:64] FLAG: --kube-api-qps="50"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740662    2580 flags.go:64] FLAG: --kube-reserved=""
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740665    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740668    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740671    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740674    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 08 20:25:26.747489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740677    2580 flags.go:64] FLAG: --lock-file=""
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740679    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740683    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740686    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740692    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740695    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740698    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740701    2580 flags.go:64] FLAG: --logging-format="text"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740704    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740707    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740710    2580 flags.go:64] FLAG: --manifest-url=""
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740713    2580 flags.go:64] FLAG: --manifest-url-header=""
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740717    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740720    2580 flags.go:64] FLAG: --max-open-files="1000000"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740724    2580 flags.go:64] FLAG: --max-pods="110"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740727    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740730    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740733    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740736    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740739    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740742    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740745    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740753    2580 flags.go:64] FLAG: --node-status-max-images="50"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740756    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740759    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 08 20:25:26.748114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740762    2580 flags.go:64] FLAG: --pod-cidr=""
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740765    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740769    2580 flags.go:64] FLAG: --pod-manifest-path=""
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740772    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740776    2580 flags.go:64] FLAG: --pods-per-core="0"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740778    2580 flags.go:64] FLAG: --port="10250"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740781    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740784    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0706dd2c1f9a836a8"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740788    2580 flags.go:64] FLAG: --qos-reserved=""
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740792    2580 flags.go:64] FLAG: --read-only-port="10255"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740796    2580 flags.go:64] FLAG: --register-node="true"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740798    2580 flags.go:64] FLAG: --register-with-taints=""
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740802    2580 flags.go:64] FLAG: --registry-burst="10"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740806    2580 flags.go:64] FLAG: --registry-qps="5"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740808    2580 flags.go:64] FLAG: --reserved-cpus=""
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740811    2580 flags.go:64] FLAG: --reserved-memory=""
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740815    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740818    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740821    2580 flags.go:64] FLAG: --rotate-certificates="false"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740823    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740826    2580 flags.go:64] FLAG: --runonce="false"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740829    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740832    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740835    2580 flags.go:64] FLAG: --seccomp-default="false"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740838    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 08 20:25:26.748720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740841    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740843    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740846    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740849    2580 flags.go:64] FLAG: --storage-driver-password="root"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740852    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740855    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740858    2580 flags.go:64] FLAG: --storage-driver-user="root"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740861    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740864    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740867    2580 flags.go:64] FLAG: --system-cgroups=""
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740869    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740875    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740877    2580 flags.go:64] FLAG: --tls-cert-file=""
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740880    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740886    2580 flags.go:64] FLAG: --tls-min-version=""
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740889    2580 flags.go:64] FLAG: --tls-private-key-file=""
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740893    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740896    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740898    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740901    2580 flags.go:64] FLAG: --v="2"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740905    2580 flags.go:64] FLAG: --version="false"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740911    2580 flags.go:64] FLAG: --vmodule=""
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740916    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.740919    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741069    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 20:25:26.749399 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741074    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741076    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741079    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741082    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741084    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741087    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741090    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741092    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741095    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741097    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741100    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741103    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741105    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741108    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741110    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741113    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741116    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741119    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741121    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741124    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741126    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 20:25:26.750080 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741129    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741131    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741136    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741157    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741161    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741164    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741167    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741170    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741175    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741177    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741181    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741186    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741190    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741193    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741196    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741199    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741202    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741205    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741207    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 20:25:26.750600 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741210    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741212    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741215    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741218    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741221    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741223    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741226    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741228    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741231    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741234    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741236    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741239    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741241    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741244    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741246    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741251    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741253    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741256    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741258    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741261    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 20:25:26.751101 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741265    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741269    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741273    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741276    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741279    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741283    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741287    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741291    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741294    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741298    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741300    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741303    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741306    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741308    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741311    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741313    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741316    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741318    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741321    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 20:25:26.751636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741324    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741326    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741329    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741331    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741334    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741337    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741339    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741342    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741346    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741349    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741351    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741354    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741356    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741359    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741363    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741365    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741369    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741371    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 20:25:26.752127 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.741374    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 20:25:26.752577 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.741379    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 20:25:26.752577 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.748334    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 08 20:25:26.752577 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.748351    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 08 20:25:26.752577 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.749084    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 08 20:25:26.752577 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.749093    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 08 20:25:26.752577 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749164    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 20:25:26.752577 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749170    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 20:25:26.752577 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749175    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 20:25:26.752577 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749178    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 20:25:26.752577 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749181    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 20:25:26.752577 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749184    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 20:25:26.752577 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749187    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 20:25:26.752577 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749190    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 20:25:26.752577 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749193    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749196    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749198    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749201    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749204    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749207    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749209    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749212    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749215    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749218    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749221    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749223    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749226    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749228    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749231    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749234    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749236    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749239    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749243    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 20:25:26.752943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749247    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749250    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749255    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749258    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749260    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749263    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749266    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749269    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749271    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749274    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749276    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749279    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749281    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749284    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749287    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749289    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749292    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749294    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749297    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749300    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 20:25:26.753437 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749303    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749305    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749308    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749310    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749313    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749315    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749318    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749320    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749323    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749326    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749328    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749331    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749334    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749337    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749340    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749343    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749346    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749349    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749352    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749354    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 20:25:26.753931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749357    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749359    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749362    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749364    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749367    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749369    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749372    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749374    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749377    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749380    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749382    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749386    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749390    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749394    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749396    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749399    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749402    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749405    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749408    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749411    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 20:25:26.754445 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749414    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 20:25:26.754940 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749417    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 20:25:26.754940 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749420    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 20:25:26.754940 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749422    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 20:25:26.754940 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749425    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 20:25:26.754940 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749429    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 20:25:26.754940 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749434    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 20:25:26.754940 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749437    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 20:25:26.754940 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749440    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 20:25:26.754940 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749443    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 20:25:26.754940 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749447    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 20:25:26.754940 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749449    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 20:25:26.754940 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.749456    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 20:25:26.754940 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749585    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 20:25:26.754940 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749590    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 20:25:26.754940 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749593    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749596    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749599    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749602    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749605    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749608    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749610    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749613    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749615    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749618    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749621    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749624    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749633    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749637    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749640    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749642    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749645    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749647    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749650    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749653    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 20:25:26.755453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749655    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749678    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749682    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749686    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749689    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749692    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749701    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749704    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749706    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749709    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749712    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749714    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749717    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749719    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749722    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749724    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749727    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749730    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749732    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749734    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 20:25:26.755937 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749737    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749739    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749742    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749744    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749747    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749749    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749752    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749754    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749757    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749759    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749762    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749764    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749767    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749770    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749774    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749777    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749780    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749783    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749786    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 20:25:26.756446 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749794    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749798    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749801    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749804    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749807    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749809    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749812    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749814    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749817    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749819    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749822    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749824    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749827    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749829    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749832    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749835    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749837    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749839    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749842    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 20:25:26.756924 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749845    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749847    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749850    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749852    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749855    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749857    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749860    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749862    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749865    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749868    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749870    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749873    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749875    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749878    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749886    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749888    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749891    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749893    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 20:25:26.757407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:26.749896    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 20:25:26.757855 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.749900    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 20:25:26.757855 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.750064    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 08 20:25:26.757855 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.753373    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 08 20:25:26.757855 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.754321    2580 server.go:1019] "Starting client certificate rotation"
Sep 08 20:25:26.757855 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.754386    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 20:25:26.757855 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.755068    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 20:25:26.757855 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.756812    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 20:25:26.765882 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.762075    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 20:25:26.772891 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.772871    2580 log.go:25] "Validated CRI v1 runtime API"
Sep 08 20:25:26.775538 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.775521    2580 log.go:25] "Validated CRI v1 image API"
Sep 08 20:25:26.776458 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.776436    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 08 20:25:26.780644 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.780613    2580 fs.go:135] Filesystem UUIDs: map[1dd375ca-2b68-43bc-a24d-00e3d9b7f2a4:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c279475e-b608-4852-86d7-d3aa94c2eb0f:/dev/nvme0n1p3]
Sep 08 20:25:26.780719 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.780643    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 08 20:25:26.784341 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.784321    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 20:25:26.786721 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.786601    2580 manager.go:216] Machine: {Timestamp:2026-09-08 20:25:26.785255985 +0000 UTC m=+0.332505458 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096130 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec228e0860f7e66a4eb40c7aa82004a8 SystemUUID:ec228e08-60f7-e66a-4eb4-0c7aa82004a8 BootID:e9fc0248-8ee2-4d17-a706-1bce69a104ab Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:16:3a:2f:33:0b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:16:3a:2f:33:0b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:9b:1e:72:4b:6a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 08 20:25:26.786721 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.786710    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 08 20:25:26.786875 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.786832    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 08 20:25:26.789454 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.789428    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 08 20:25:26.789598 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.789457    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-142.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 08 20:25:26.790077 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.790066    2580 topology_manager.go:138] "Creating topology manager with none policy"
Sep 08 20:25:26.790115 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.790080    2580 container_manager_linux.go:309] "Creating device plugin manager"
Sep 08 20:25:26.790115 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.790088    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 20:25:26.790713 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.790703    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 20:25:26.792086 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.792076    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 08 20:25:26.795461 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.795447    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 20:25:26.795579 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.795570    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 08 20:25:26.795667 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.795600    2580 kubelet.go:486] "Attempting to sync node with API server"
Sep 08 20:25:26.795667 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.795610    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 08 20:25:26.795667 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.795623    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 08 20:25:26.795667 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.795633    2580 kubelet.go:398] "Adding apiserver pod source"
Sep 08 20:25:26.795667 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.795644    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 08 20:25:26.796140 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.796126    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 20:25:26.796197 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.796150    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 20:25:26.798759 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.798737    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 08 20:25:26.800899 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.800884    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 08 20:25:26.800988 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.800914    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 08 20:25:26.801698 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.801685    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 08 20:25:26.801746 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.801707    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 08 20:25:26.801746 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.801713    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 08 20:25:26.801746 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.801725    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 08 20:25:26.801746 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.801733    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 08 20:25:26.801746 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.801739    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 08 20:25:26.801885 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.801753    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 08 20:25:26.801885 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.801758    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 08 20:25:26.801885 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.801765    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 08 20:25:26.801885 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.801770    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 08 20:25:26.801885 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.801785    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 08 20:25:26.801885 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.801791    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 08 20:25:26.802542 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.802532    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 08 20:25:26.802542 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.802542    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 08 20:25:26.806000 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.805980    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-142.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 08 20:25:26.806000 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:26.805989    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 08 20:25:26.806096 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:26.806015    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-142.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 08 20:25:26.806657 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.806645    2580 server.go:1268] "Started kubelet"
Sep 08 20:25:26.806816 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.806786    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 08 20:25:26.806884 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.806837    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 08 20:25:26.807278 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.807229    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 08 20:25:26.808006 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.807986    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 08 20:25:26.808065 ip-10-0-136-142 systemd[1]: Started Kubernetes Kubelet.
Sep 08 20:25:26.813371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.813346    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cjrdf"
Sep 08 20:25:26.813769 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:26.813739    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 08 20:25:26.814972 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.814940    2580 server.go:310] "Adding debug handlers to kubelet server"
Sep 08 20:25:26.816330 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:26.815015    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-142.ec2.internal.18d37247205097d8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-142.ec2.internal,UID:ip-10-0-136-142.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-142.ec2.internal,},FirstTimestamp:2026-09-08 20:25:26.806616024 +0000 UTC m=+0.353865495,LastTimestamp:2026-09-08 20:25:26.806616024 +0000 UTC m=+0.353865495,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-142.ec2.internal,}"
Sep 08 20:25:26.818941 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.818921    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 08 20:25:26.819375 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.819363    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 08 20:25:26.819464 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:26.819445    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-142.ec2.internal\" not found"
Sep 08 20:25:26.820128 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.820110    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 08 20:25:26.820222 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.820114    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 08 20:25:26.820222 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.820149    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 08 20:25:26.820222 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.820201    2580 factory.go:55] Registering systemd factory
Sep 08 20:25:26.820222 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.820219    2580 factory.go:223] Registration of the systemd container factory successfully
Sep 08 20:25:26.820389 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.820235    2580 reconstruct.go:97] "Volume reconstruction finished"
Sep 08 20:25:26.820389 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.820241    2580 reconciler.go:29] "Reconciler: start to sync state"
Sep 08 20:25:26.820474 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.820451    2580 factory.go:153] Registering CRI-O factory
Sep 08 20:25:26.820474 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.820465    2580 factory.go:223] Registration of the crio container factory successfully
Sep 08 20:25:26.820543 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.820516    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 08 20:25:26.820543 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.820534    2580 factory.go:103] Registering Raw factory
Sep 08 20:25:26.820661 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.820569    2580 manager.go:1196] Started watching for new ooms in manager
Sep 08 20:25:26.820992 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.820979    2580 manager.go:349] Starting recovery of all containers
Sep 08 20:25:26.822025 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.821999    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cjrdf"
Sep 08 20:25:26.829747 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.829603    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 20:25:26.830008 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.829992    2580 manager.go:354] Recovery completed
Sep 08 20:25:26.832723 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:26.832699    2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-142.ec2.internal\" not found" node="ip-10-0-136-142.ec2.internal"
Sep 08 20:25:26.835172 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.835159    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 20:25:26.837030 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.837017    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 20:25:26.837118 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.837056    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 20:25:26.837118 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.837073    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasSufficientPID"
Sep 08 20:25:26.837812 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.837795    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 08 20:25:26.837812 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.837809    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 08 20:25:26.837936 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.837827    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 20:25:26.837936 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.837799    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 20:25:26.838663 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.838636    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 20:25:26.838747 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.838672    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 20:25:26.838747 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.838705    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasSufficientPID"
Sep 08 20:25:26.839867 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.839851    2580 policy_none.go:49] "None policy: Start"
Sep 08 20:25:26.839867 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.839871    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 08 20:25:26.839998 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.839884    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 08 20:25:26.841773 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.841757    2580 policy_none.go:47] "Start"
Sep 08 20:25:26.869658 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.869638    2580 manager.go:337] "Starting Device Plugin manager"
Sep 08 20:25:26.896358 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:26.869685    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 08 20:25:26.896358 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.869698    2580 server.go:87] "Starting device plugin registration server"
Sep 08 20:25:26.896358 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.870094    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 08 20:25:26.896358 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.870107    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 08 20:25:26.896358 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.870208    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 08 20:25:26.896358 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.870295    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 08 20:25:26.896358 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.870305    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 08 20:25:26.896358 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:26.870900    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 08 20:25:26.896358 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:26.870946    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-142.ec2.internal\" not found"
Sep 08 20:25:26.915768 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.915734    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 08 20:25:26.916999 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.916970    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 08 20:25:26.917097 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.917043    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 08 20:25:26.917097 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.917076    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 08 20:25:26.917247 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:26.917184    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 08 20:25:26.920139 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.920116    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 20:25:26.970535 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.970505    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 20:25:26.971726 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.971710    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 20:25:26.971798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.971745    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 20:25:26.971798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.971760    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasSufficientPID"
Sep 08 20:25:26.971798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.971789    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-142.ec2.internal"
Sep 08 20:25:26.978689 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:26.978667    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-142.ec2.internal"
Sep 08 20:25:26.978798 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:26.978695    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-142.ec2.internal\": node \"ip-10-0-136-142.ec2.internal\" not found"
Sep 08 20:25:27.017888 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.017779    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-142.ec2.internal"]
Sep 08 20:25:27.017888 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.017868    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 20:25:27.018897 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.018880    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 20:25:27.019005 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.018916    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 20:25:27.019005 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.018930    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasSufficientPID"
Sep 08 20:25:27.023227 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:27.023209    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-142.ec2.internal\" not found"
Sep 08 20:25:27.023718 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.023703    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 20:25:27.023853 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.023837    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal"
Sep 08 20:25:27.023914 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.023867    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 20:25:27.024567 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.024544    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 20:25:27.024666 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.024580    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 20:25:27.024666 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.024592    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasSufficientPID"
Sep 08 20:25:27.024666 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.024595    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 20:25:27.024666 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.024614    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 20:25:27.024666 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.024622    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasSufficientPID"
Sep 08 20:25:27.027547 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.027529    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-142.ec2.internal"
Sep 08 20:25:27.027637 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.027556    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 20:25:27.028302 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.028285    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 20:25:27.028376 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.028313    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 20:25:27.028376 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.028325    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeHasSufficientPID"
Sep 08 20:25:27.049806 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:27.049781    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-142.ec2.internal\" not found" node="ip-10-0-136-142.ec2.internal"
Sep 08 20:25:27.054416 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:27.054397    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-142.ec2.internal\" not found" node="ip-10-0-136-142.ec2.internal"
Sep 08 20:25:27.121189 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.121153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e47a49cba2d858e2e9885f9d721c4db-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal\" (UID: \"1e47a49cba2d858e2e9885f9d721c4db\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal"
Sep 08 20:25:27.121349 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.121200    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e47a49cba2d858e2e9885f9d721c4db-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal\" (UID: \"1e47a49cba2d858e2e9885f9d721c4db\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal"
Sep 08 20:25:27.124244 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:27.124227    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-142.ec2.internal\" not found"
Sep 08 20:25:27.221392 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.221358    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e47a49cba2d858e2e9885f9d721c4db-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal\" (UID: \"1e47a49cba2d858e2e9885f9d721c4db\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal"
Sep 08 20:25:27.221392 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.221391    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e47a49cba2d858e2e9885f9d721c4db-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal\" (UID: \"1e47a49cba2d858e2e9885f9d721c4db\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal"
Sep 08 20:25:27.221582 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.221413    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4dade65b3f03b9eb8dc1b57ffacbb7fd-config\") pod \"kube-apiserver-proxy-ip-10-0-136-142.ec2.internal\" (UID: \"4dade65b3f03b9eb8dc1b57ffacbb7fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-142.ec2.internal"
Sep 08 20:25:27.221582 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.221464    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e47a49cba2d858e2e9885f9d721c4db-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal\" (UID: \"1e47a49cba2d858e2e9885f9d721c4db\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal"
Sep 08 20:25:27.221582 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.221477    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e47a49cba2d858e2e9885f9d721c4db-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal\" (UID: \"1e47a49cba2d858e2e9885f9d721c4db\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal"
Sep 08 20:25:27.224469 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:27.224451    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-142.ec2.internal\" not found"
Sep 08 20:25:27.322210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.322131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4dade65b3f03b9eb8dc1b57ffacbb7fd-config\") pod \"kube-apiserver-proxy-ip-10-0-136-142.ec2.internal\" (UID: \"4dade65b3f03b9eb8dc1b57ffacbb7fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-142.ec2.internal"
Sep 08 20:25:27.322210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.322198    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4dade65b3f03b9eb8dc1b57ffacbb7fd-config\") pod \"kube-apiserver-proxy-ip-10-0-136-142.ec2.internal\" (UID: \"4dade65b3f03b9eb8dc1b57ffacbb7fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-142.ec2.internal"
Sep 08 20:25:27.325244 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:27.325225    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-142.ec2.internal\" not found"
Sep 08 20:25:27.353429 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.353401    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal"
Sep 08 20:25:27.356774 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.356746    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-142.ec2.internal"
Sep 08 20:25:27.426190 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:27.426152    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-142.ec2.internal\" not found"
Sep 08 20:25:27.526748 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:27.526719    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-142.ec2.internal\" not found"
Sep 08 20:25:27.627364 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:27.627290    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-142.ec2.internal\" not found"
Sep 08 20:25:27.728220 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:27.728176    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-142.ec2.internal\" not found"
Sep 08 20:25:27.754647 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.754615    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 08 20:25:27.755109 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.754800    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 20:25:27.755109 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.754823    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 20:25:27.825487 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.825447    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 20:20:27 +0000 UTC" deadline="2028-05-18 20:51:38.236129596 +0000 UTC"
Sep 08 20:25:27.825487 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:27.825480    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14832h26m10.410653078s"
Sep 08 20:25:27.828556 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:27.828539    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-142.ec2.internal\" not found"
Sep 08 20:25:27.929229 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:27.929144    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-142.ec2.internal\" not found"
Sep 08 20:25:28.029819 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:28.029784    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-142.ec2.internal\" not found"
Sep 08 20:25:28.073722 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:28.073674    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e47a49cba2d858e2e9885f9d721c4db.slice/crio-4c575acc6120ce65b9400e5a4fb692d80860c84eb192b063b0de6781b8c5a0ed WatchSource:0}: Error finding container 4c575acc6120ce65b9400e5a4fb692d80860c84eb192b063b0de6781b8c5a0ed: Status 404 returned error can't find the container with id 4c575acc6120ce65b9400e5a4fb692d80860c84eb192b063b0de6781b8c5a0ed
Sep 08 20:25:28.073955 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:28.073927    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dade65b3f03b9eb8dc1b57ffacbb7fd.slice/crio-8a4c0ee22f61252acdb6b318d1da42928ffe230347b57f22891c739802d12c74 WatchSource:0}: Error finding container 8a4c0ee22f61252acdb6b318d1da42928ffe230347b57f22891c739802d12c74: Status 404 returned error can't find the container with id 8a4c0ee22f61252acdb6b318d1da42928ffe230347b57f22891c739802d12c74
Sep 08 20:25:28.079975 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.079957    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 20:25:28.130226 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:28.130172    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-142.ec2.internal\" not found"
Sep 08 20:25:28.214764 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.214700    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 20:25:28.219856 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.219838    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal"
Sep 08 20:25:28.232024 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.231997    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 20:25:28.234060 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.234041    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-142.ec2.internal"
Sep 08 20:25:28.243877 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.243858    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 20:25:28.311168 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.311139    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 20:25:28.796773 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.796728    2580 apiserver.go:52] "Watching apiserver"
Sep 08 20:25:28.802881 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.802846    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 08 20:25:28.805337 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.805303    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-142.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx","openshift-cluster-node-tuning-operator/tuned-8h7mp","openshift-network-diagnostics/network-check-target-vjhfp","openshift-ovn-kubernetes/ovnkube-node-x29fz","openshift-dns/node-resolver-cljhb","openshift-image-registry/node-ca-gc9sv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal","openshift-multus/multus-additional-cni-plugins-gnm8w","openshift-multus/multus-bghvs","openshift-multus/network-metrics-daemon-vlx4d","openshift-network-operator/iptables-alerter-dpq9v","kube-system/global-pull-secret-syncer-fkjc4","kube-system/konnectivity-agent-g26sh"]
Sep 08 20:25:28.807840 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.807811    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gc9sv"
Sep 08 20:25:28.810213 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.810190    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:28.810343 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:28.810274    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:25:28.811063 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.810748    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 08 20:25:28.811063 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.810799    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 08 20:25:28.811063 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.810861    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-59rt8\""
Sep 08 20:25:28.811063 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.810799    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 08 20:25:28.821550 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.817191    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.821550 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.820155    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 08 20:25:28.821550 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.820187    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-j5d7h\""
Sep 08 20:25:28.821550 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.820353    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 08 20:25:28.821550 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.821104    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 08 20:25:28.826713 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.826660    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:28.826984 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:28.826958    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:28.829818 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.829153    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.831561 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.831518    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 08 20:25:28.831883 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.831865    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 08 20:25:28.832411 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.832369    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 08 20:25:28.832740 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.832721    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 08 20:25:28.834392 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.834366    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-host\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.834498 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.834408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b691d62f-ea1a-43a9-909e-ab03a8bf9435-host\") pod \"node-ca-gc9sv\" (UID: \"b691d62f-ea1a-43a9-909e-ab03a8bf9435\") " pod="openshift-image-registry/node-ca-gc9sv"
Sep 08 20:25:28.834498 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.834005    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 08 20:25:28.834498 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.834159    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 08 20:25:28.834709 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.834266    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cljhb"
Sep 08 20:25:28.834709 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.834235    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zhrfm\""
Sep 08 20:25:28.835087 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.834440    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5dk5\" (UniqueName: \"kubernetes.io/projected/b691d62f-ea1a-43a9-909e-ab03a8bf9435-kube-api-access-l5dk5\") pod \"node-ca-gc9sv\" (UID: \"b691d62f-ea1a-43a9-909e-ab03a8bf9435\") " pod="openshift-image-registry/node-ca-gc9sv"
Sep 08 20:25:28.835175 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835140    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-modprobe-d\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.835231 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835190    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-sys\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.835231 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835220    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lck5b\" (UniqueName: \"kubernetes.io/projected/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-kube-api-access-lck5b\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.835336 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835254    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-sysctl-d\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.835336 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835283    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b691d62f-ea1a-43a9-909e-ab03a8bf9435-serviceca\") pod \"node-ca-gc9sv\" (UID: \"b691d62f-ea1a-43a9-909e-ab03a8bf9435\") " pod="openshift-image-registry/node-ca-gc9sv"
Sep 08 20:25:28.835336 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-sysconfig\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.835480 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835336    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-tmp\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.835480 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdnvw\" (UniqueName: \"kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw\") pod \"network-check-target-vjhfp\" (UID: \"91b9ef98-2cf6-4b27-97b5-01662e8c4058\") " pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:28.835480 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret\") pod \"global-pull-secret-syncer-fkjc4\" (UID: \"a9304897-72a4-4daa-aa6e-9397a440502e\") " pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:28.835480 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835435    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-sysctl-conf\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.835686 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-lib-modules\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.835686 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-var-lib-kubelet\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.835686 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835543    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-tuned\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.835686 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a9304897-72a4-4daa-aa6e-9397a440502e-kubelet-config\") pod \"global-pull-secret-syncer-fkjc4\" (UID: \"a9304897-72a4-4daa-aa6e-9397a440502e\") " pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:28.835686 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a9304897-72a4-4daa-aa6e-9397a440502e-dbus\") pod \"global-pull-secret-syncer-fkjc4\" (UID: \"a9304897-72a4-4daa-aa6e-9397a440502e\") " pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:28.835686 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835619    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-kubernetes\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.835686 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835642    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-systemd\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.835686 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-run\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.836109 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.835898    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:28.837233 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.836982    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 20:25:28.837233 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.837053    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-t2w4f\""
Sep 08 20:25:28.837233 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.837194    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 08 20:25:28.837233 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.837224    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 08 20:25:28.838362 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.837821    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g26sh"
Sep 08 20:25:28.839268 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.838551    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 08 20:25:28.839268 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.838807    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 08 20:25:28.839268 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.838921    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 08 20:25:28.839268 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.838977    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 08 20:25:28.839671 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.839592    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 08 20:25:28.839889 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.839868    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8nls4\""
Sep 08 20:25:28.839987 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.839970    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 08 20:25:28.840472 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.840452    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dg2vt\""
Sep 08 20:25:28.841058 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.841040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:28.841619 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.841601    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 08 20:25:28.843264 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.843242    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 08 20:25:28.843622 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.843602    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 08 20:25:28.843855 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.843837    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 08 20:25:28.844094 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.844076    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5xwfw\""
Sep 08 20:25:28.844170 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.844102    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.847021 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.847001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:28.847139 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:28.847091    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:28.848592 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.848569    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7vdsn\""
Sep 08 20:25:28.848922 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.848901    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 08 20:25:28.849396 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.849353    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dpq9v"
Sep 08 20:25:28.851621 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.851600    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hjnw4\""
Sep 08 20:25:28.852026 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.852003    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 08 20:25:28.852327 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.852304    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 08 20:25:28.852828 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.852807    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 08 20:25:28.860713 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.860691    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-t8hgb"
Sep 08 20:25:28.870375 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.870348    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-t8hgb"
Sep 08 20:25:28.921461 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.921431    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 08 20:25:28.925792 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.925736    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal" event={"ID":"1e47a49cba2d858e2e9885f9d721c4db","Type":"ContainerStarted","Data":"4c575acc6120ce65b9400e5a4fb692d80860c84eb192b063b0de6781b8c5a0ed"}
Sep 08 20:25:28.929070 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.928596    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-142.ec2.internal" event={"ID":"4dade65b3f03b9eb8dc1b57ffacbb7fd","Type":"ContainerStarted","Data":"8a4c0ee22f61252acdb6b318d1da42928ffe230347b57f22891c739802d12c74"}
Sep 08 20:25:28.936513 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdnvw\" (UniqueName: \"kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw\") pod \"network-check-target-vjhfp\" (UID: \"91b9ef98-2cf6-4b27-97b5-01662e8c4058\") " pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:28.936652 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/de02ff1a-312a-4069-b121-dff1aa3b9354-hosts-file\") pod \"node-resolver-cljhb\" (UID: \"de02ff1a-312a-4069-b121-dff1aa3b9354\") " pod="openshift-dns/node-resolver-cljhb"
Sep 08 20:25:28.936652 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936561    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ddd3b60-46d5-4acf-a364-110c0486e506-cni-binary-copy\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.936652 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-multus-conf-dir\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.936652 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936614    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-var-lib-openvswitch\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.936652 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936641    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-registration-dir\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:28.936920 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-sysctl-conf\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.936920 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936697    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-var-lib-kubelet\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.936920 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936771    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-var-lib-kubelet\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.936920 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/15838251-a64d-4585-831a-69d5bb59336b-konnectivity-ca\") pod \"konnectivity-agent-g26sh\" (UID: \"15838251-a64d-4585-831a-69d5bb59336b\") " pod="kube-system/konnectivity-agent-g26sh"
Sep 08 20:25:28.936920 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-run-netns\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.936920 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936847    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-run-multus-certs\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.936920 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936872    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-etc-kubernetes\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.936920 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936875    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-sysctl-conf\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.936920 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936904    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-run-netns\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.937328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936936    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kblr2\" (UniqueName: \"kubernetes.io/projected/daf93c03-ec62-42a5-90e3-ae79820e863f-kube-api-access-kblr2\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.937328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.936985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-run\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.937328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937014    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-multus-cni-dir\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.937328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937040    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-run-ovn-kubernetes\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.937328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937065    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-run\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.937328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937066    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-system-cni-dir\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:28.937328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937123    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjzc6\" (UniqueName: \"kubernetes.io/projected/642122c1-c2a9-4fe3-aade-12ac05cecf17-kube-api-access-bjzc6\") pod \"network-metrics-daemon-vlx4d\" (UID: \"642122c1-c2a9-4fe3-aade-12ac05cecf17\") " pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:28.937328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937149    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/daf93c03-ec62-42a5-90e3-ae79820e863f-ovn-node-metrics-cert\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.937328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937167    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-cni-binary-copy\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:28.937328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937183    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8dvl\" (UniqueName: \"kubernetes.io/projected/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-kube-api-access-k8dvl\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:28.937328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937200    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-etc-selinux\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:28.937328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret\") pod \"global-pull-secret-syncer-fkjc4\" (UID: \"a9304897-72a4-4daa-aa6e-9397a440502e\") " pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:28.937328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937281    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-var-lib-kubelet\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.937328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937307    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-kubelet\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.937328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937332    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:28.937956 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:28.937357    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:28.937956 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937358    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/15838251-a64d-4585-831a-69d5bb59336b-agent-certs\") pod \"konnectivity-agent-g26sh\" (UID: \"15838251-a64d-4585-831a-69d5bb59336b\") " pod="kube-system/konnectivity-agent-g26sh"
Sep 08 20:25:28.937956 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:28.937421    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret podName:a9304897-72a4-4daa-aa6e-9397a440502e nodeName:}" failed. No retries permitted until 2026-09-08 20:25:29.43740483 +0000 UTC m=+2.984654290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret") pod "global-pull-secret-syncer-fkjc4" (UID: "a9304897-72a4-4daa-aa6e-9397a440502e") : object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:28.937956 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a9304897-72a4-4daa-aa6e-9397a440502e-dbus\") pod \"global-pull-secret-syncer-fkjc4\" (UID: \"a9304897-72a4-4daa-aa6e-9397a440502e\") " pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:28.937956 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937511    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-system-cni-dir\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.937956 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937537    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/daf93c03-ec62-42a5-90e3-ae79820e863f-ovnkube-script-lib\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.937956 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937582    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-cnibin\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:28.937956 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937608    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:28.937956 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a9304897-72a4-4daa-aa6e-9397a440502e-dbus\") pod \"global-pull-secret-syncer-fkjc4\" (UID: \"a9304897-72a4-4daa-aa6e-9397a440502e\") " pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:28.937956 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5dk5\" (UniqueName: \"kubernetes.io/projected/b691d62f-ea1a-43a9-909e-ab03a8bf9435-kube-api-access-l5dk5\") pod \"node-ca-gc9sv\" (UID: \"b691d62f-ea1a-43a9-909e-ab03a8bf9435\") " pod="openshift-image-registry/node-ca-gc9sv"
Sep 08 20:25:28.937956 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937672    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-multus-socket-dir-parent\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.937956 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-sys-fs\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:28.937956 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fc3088bd-7250-4a7a-bf55-432996628b80-iptables-alerter-script\") pod \"iptables-alerter-dpq9v\" (UID: \"fc3088bd-7250-4a7a-bf55-432996628b80\") " pod="openshift-network-operator/iptables-alerter-dpq9v"
Sep 08 20:25:28.937956 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpkk5\" (UniqueName: \"kubernetes.io/projected/fc3088bd-7250-4a7a-bf55-432996628b80-kube-api-access-wpkk5\") pod \"iptables-alerter-dpq9v\" (UID: \"fc3088bd-7250-4a7a-bf55-432996628b80\") " pod="openshift-network-operator/iptables-alerter-dpq9v"
Sep 08 20:25:28.937956 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b691d62f-ea1a-43a9-909e-ab03a8bf9435-serviceca\") pod \"node-ca-gc9sv\" (UID: \"b691d62f-ea1a-43a9-909e-ab03a8bf9435\") " pod="openshift-image-registry/node-ca-gc9sv"
Sep 08 20:25:28.937956 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937811    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-cnibin\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937835    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-var-lib-cni-multus\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-run-systemd\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937900    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/daf93c03-ec62-42a5-90e3-ae79820e863f-env-overrides\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.937973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-run-k8s-cni-cncf-io\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-slash\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-device-dir\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938108    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbcl6\" (UniqueName: \"kubernetes.io/projected/de02ff1a-312a-4069-b121-dff1aa3b9354-kube-api-access-lbcl6\") pod \"node-resolver-cljhb\" (UID: \"de02ff1a-312a-4069-b121-dff1aa3b9354\") " pod="openshift-dns/node-resolver-cljhb"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938155    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-systemd\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938208    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-host\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938219    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-systemd\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938236    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-cni-netd\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-socket-dir\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938298    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmjfv\" (UniqueName: \"kubernetes.io/projected/ae6a91db-aacf-4fea-8169-a33821987ca7-kube-api-access-jmjfv\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938271    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-host\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938330    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lck5b\" (UniqueName: \"kubernetes.io/projected/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-kube-api-access-lck5b\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.938797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938369    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-var-lib-cni-bin\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938394    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-log-socket\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-sysconfig\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ddd3b60-46d5-4acf-a364-110c0486e506-multus-daemon-config\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938470    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-run-ovn\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938497    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938513    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-sysconfig\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938525    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-lib-modules\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938596    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-tuned\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938626    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xttd6\" (UniqueName: \"kubernetes.io/projected/1ddd3b60-46d5-4acf-a364-110c0486e506-kube-api-access-xttd6\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938666    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-etc-openvswitch\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938691    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs\") pod \"network-metrics-daemon-vlx4d\" (UID: \"642122c1-c2a9-4fe3-aade-12ac05cecf17\") " pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938712    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc3088bd-7250-4a7a-bf55-432996628b80-host-slash\") pod \"iptables-alerter-dpq9v\" (UID: \"fc3088bd-7250-4a7a-bf55-432996628b80\") " pod="openshift-network-operator/iptables-alerter-dpq9v"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a9304897-72a4-4daa-aa6e-9397a440502e-kubelet-config\") pod \"global-pull-secret-syncer-fkjc4\" (UID: \"a9304897-72a4-4daa-aa6e-9397a440502e\") " pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-kubernetes\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938780    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de02ff1a-312a-4069-b121-dff1aa3b9354-tmp-dir\") pod \"node-resolver-cljhb\" (UID: \"de02ff1a-312a-4069-b121-dff1aa3b9354\") " pod="openshift-dns/node-resolver-cljhb"
Sep 08 20:25:28.939384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938818    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-lib-modules\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938820    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-node-log\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938886    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a9304897-72a4-4daa-aa6e-9397a440502e-kubelet-config\") pod \"global-pull-secret-syncer-fkjc4\" (UID: \"a9304897-72a4-4daa-aa6e-9397a440502e\") " pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938930    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-kubernetes\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.938971    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-cni-bin\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.939013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b691d62f-ea1a-43a9-909e-ab03a8bf9435-host\") pod \"node-ca-gc9sv\" (UID: \"b691d62f-ea1a-43a9-909e-ab03a8bf9435\") " pod="openshift-image-registry/node-ca-gc9sv"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.939046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-modprobe-d\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.939055    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b691d62f-ea1a-43a9-909e-ab03a8bf9435-serviceca\") pod \"node-ca-gc9sv\" (UID: \"b691d62f-ea1a-43a9-909e-ab03a8bf9435\") " pod="openshift-image-registry/node-ca-gc9sv"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.939073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-sys\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.939072    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b691d62f-ea1a-43a9-909e-ab03a8bf9435-host\") pod \"node-ca-gc9sv\" (UID: \"b691d62f-ea1a-43a9-909e-ab03a8bf9435\") " pod="openshift-image-registry/node-ca-gc9sv"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.939130    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-sys\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.939131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-hostroot\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.939159    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-modprobe-d\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.939177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-systemd-units\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.939205    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-run-openvswitch\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.939231    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/daf93c03-ec62-42a5-90e3-ae79820e863f-ovnkube-config\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.939259    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-sysctl-d\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.939282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-tmp\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.940313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.939365    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 08 20:25:28.941126 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.939440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-sysctl-d\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.943961 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:28.943936    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 20:25:28.944102 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:28.943986    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 20:25:28.944102 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:28.944002    2580 projected.go:196] Error preparing data for projected volume kube-api-access-jdnvw for pod openshift-network-diagnostics/network-check-target-vjhfp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:28.945600 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:28.944115    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw podName:91b9ef98-2cf6-4b27-97b5-01662e8c4058 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:29.444084806 +0000 UTC m=+2.991334284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jdnvw" (UniqueName: "kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw") pod "network-check-target-vjhfp" (UID: "91b9ef98-2cf6-4b27-97b5-01662e8c4058") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:28.945600 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.944547    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-tmp\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.945600 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.944576    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-etc-tuned\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:28.947674 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.947645    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5dk5\" (UniqueName: \"kubernetes.io/projected/b691d62f-ea1a-43a9-909e-ab03a8bf9435-kube-api-access-l5dk5\") pod \"node-ca-gc9sv\" (UID: \"b691d62f-ea1a-43a9-909e-ab03a8bf9435\") " pod="openshift-image-registry/node-ca-gc9sv"
Sep 08 20:25:28.950452 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:28.950365    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lck5b\" (UniqueName: \"kubernetes.io/projected/5a574ef3-d788-41e3-bdc3-2e2bd3e10621-kube-api-access-lck5b\") pod \"tuned-8h7mp\" (UID: \"5a574ef3-d788-41e3-bdc3-2e2bd3e10621\") " pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:29.040373 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040335    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-hostroot\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.040541 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-systemd-units\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.040541 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040408    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-run-openvswitch\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.040541 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040459    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-systemd-units\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.040541 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040489    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-run-openvswitch\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.040541 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040459    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-hostroot\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.040541 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040462    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/daf93c03-ec62-42a5-90e3-ae79820e863f-ovnkube-config\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.040737 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/de02ff1a-312a-4069-b121-dff1aa3b9354-hosts-file\") pod \"node-resolver-cljhb\" (UID: \"de02ff1a-312a-4069-b121-dff1aa3b9354\") " pod="openshift-dns/node-resolver-cljhb"
Sep 08 20:25:29.040737 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ddd3b60-46d5-4acf-a364-110c0486e506-cni-binary-copy\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.040737 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-multus-conf-dir\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.040737 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-var-lib-openvswitch\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.040737 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-registration-dir\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:29.040737 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040635    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/de02ff1a-312a-4069-b121-dff1aa3b9354-hosts-file\") pod \"node-resolver-cljhb\" (UID: \"de02ff1a-312a-4069-b121-dff1aa3b9354\") " pod="openshift-dns/node-resolver-cljhb"
Sep 08 20:25:29.040737 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040672    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-multus-conf-dir\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.040737 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040720    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-var-lib-openvswitch\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.040989 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/15838251-a64d-4585-831a-69d5bb59336b-konnectivity-ca\") pod \"konnectivity-agent-g26sh\" (UID: \"15838251-a64d-4585-831a-69d5bb59336b\") " pod="kube-system/konnectivity-agent-g26sh"
Sep 08 20:25:29.040989 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-registration-dir\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:29.040989 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-run-netns\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.040989 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040890    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-run-multus-certs\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.040989 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040917    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-etc-kubernetes\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.040989 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040946    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-run-netns\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.040989 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.040972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kblr2\" (UniqueName: \"kubernetes.io/projected/daf93c03-ec62-42a5-90e3-ae79820e863f-kube-api-access-kblr2\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.041314 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041015    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-multus-cni-dir\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.041314 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-run-ovn-kubernetes\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.041314 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041065    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-etc-kubernetes\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.041314 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-system-cni-dir\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:29.041314 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041111    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-run-netns\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.041314 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041157    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/daf93c03-ec62-42a5-90e3-ae79820e863f-ovnkube-config\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.041314 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-system-cni-dir\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:29.041314 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041234    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-run-ovn-kubernetes\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.041314 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041241    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-run-netns\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.041314 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041271    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-run-multus-certs\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.041314 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041272    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ddd3b60-46d5-4acf-a364-110c0486e506-cni-binary-copy\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.041314 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041272    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/15838251-a64d-4585-831a-69d5bb59336b-konnectivity-ca\") pod \"konnectivity-agent-g26sh\" (UID: \"15838251-a64d-4585-831a-69d5bb59336b\") " pod="kube-system/konnectivity-agent-g26sh"
Sep 08 20:25:29.041314 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041312    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjzc6\" (UniqueName: \"kubernetes.io/projected/642122c1-c2a9-4fe3-aade-12ac05cecf17-kube-api-access-bjzc6\") pod \"network-metrics-daemon-vlx4d\" (UID: \"642122c1-c2a9-4fe3-aade-12ac05cecf17\") " pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041347    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/daf93c03-ec62-42a5-90e3-ae79820e863f-ovn-node-metrics-cert\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041402    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-multus-cni-dir\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041425    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-cni-binary-copy\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041514    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8dvl\" (UniqueName: \"kubernetes.io/projected/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-kube-api-access-k8dvl\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041558    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-etc-selinux\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041600    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-var-lib-kubelet\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041626    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-kubelet\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/15838251-a64d-4585-831a-69d5bb59336b-agent-certs\") pod \"konnectivity-agent-g26sh\" (UID: \"15838251-a64d-4585-831a-69d5bb59336b\") " pod="kube-system/konnectivity-agent-g26sh"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041727    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-kubelet\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041747    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-etc-selinux\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041784    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-var-lib-kubelet\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-system-cni-dir\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041816    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-system-cni-dir\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041802    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/daf93c03-ec62-42a5-90e3-ae79820e863f-ovnkube-script-lib\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.041978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-cnibin\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-multus-socket-dir-parent\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041960    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-sys-fs\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042006    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fc3088bd-7250-4a7a-bf55-432996628b80-iptables-alerter-script\") pod \"iptables-alerter-dpq9v\" (UID: \"fc3088bd-7250-4a7a-bf55-432996628b80\") " pod="openshift-network-operator/iptables-alerter-dpq9v"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042056    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpkk5\" (UniqueName: \"kubernetes.io/projected/fc3088bd-7250-4a7a-bf55-432996628b80-kube-api-access-wpkk5\") pod \"iptables-alerter-dpq9v\" (UID: \"fc3088bd-7250-4a7a-bf55-432996628b80\") " pod="openshift-network-operator/iptables-alerter-dpq9v"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042093    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-cnibin\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-var-lib-cni-multus\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042140    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-sys-fs\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-run-systemd\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042194    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-multus-socket-dir-parent\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042200    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/daf93c03-ec62-42a5-90e3-ae79820e863f-env-overrides\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042227    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042248    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-cnibin\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-run-k8s-cni-cncf-io\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042277    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-slash\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.042762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042299    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-device-dir\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbcl6\" (UniqueName: \"kubernetes.io/projected/de02ff1a-312a-4069-b121-dff1aa3b9354-kube-api-access-lbcl6\") pod \"node-resolver-cljhb\" (UID: \"de02ff1a-312a-4069-b121-dff1aa3b9354\") " pod="openshift-dns/node-resolver-cljhb"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042358    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/daf93c03-ec62-42a5-90e3-ae79820e863f-ovnkube-script-lib\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-cni-netd\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042412    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-socket-dir\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmjfv\" (UniqueName: \"kubernetes.io/projected/ae6a91db-aacf-4fea-8169-a33821987ca7-kube-api-access-jmjfv\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-var-lib-cni-bin\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-log-socket\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ddd3b60-46d5-4acf-a364-110c0486e506-multus-daemon-config\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-run-ovn\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042601    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042613    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-device-dir\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042666    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-var-lib-cni-multus\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042679    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xttd6\" (UniqueName: \"kubernetes.io/projected/1ddd3b60-46d5-4acf-a364-110c0486e506-kube-api-access-xttd6\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-run-systemd\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-etc-openvswitch\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.043509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fc3088bd-7250-4a7a-bf55-432996628b80-iptables-alerter-script\") pod \"iptables-alerter-dpq9v\" (UID: \"fc3088bd-7250-4a7a-bf55-432996628b80\") " pod="openshift-network-operator/iptables-alerter-dpq9v"
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs\") pod \"network-metrics-daemon-vlx4d\" (UID: \"642122c1-c2a9-4fe3-aade-12ac05cecf17\") " pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042774    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-etc-openvswitch\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc3088bd-7250-4a7a-bf55-432996628b80-host-slash\") pod \"iptables-alerter-dpq9v\" (UID: \"fc3088bd-7250-4a7a-bf55-432996628b80\") " pod="openshift-network-operator/iptables-alerter-dpq9v"
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042827    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de02ff1a-312a-4069-b121-dff1aa3b9354-tmp-dir\") pod \"node-resolver-cljhb\" (UID: \"de02ff1a-312a-4069-b121-dff1aa3b9354\") " pod="openshift-dns/node-resolver-cljhb"
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042852    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-node-log\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042878    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-cni-bin\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.042977    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-cni-bin\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.043020    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-run-k8s-cni-cncf-io\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.043069    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-cni-netd\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.043073    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-slash\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:29.043210    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.043226    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-cni-binary-copy\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:29.043279    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs podName:642122c1-c2a9-4fe3-aade-12ac05cecf17 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:29.543249136 +0000 UTC m=+3.090498598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs") pod "network-metrics-daemon-vlx4d" (UID: "642122c1-c2a9-4fe3-aade-12ac05cecf17") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.043282    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.043294    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-node-log\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.043335    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc3088bd-7250-4a7a-bf55-432996628b80-host-slash\") pod \"iptables-alerter-dpq9v\" (UID: \"fc3088bd-7250-4a7a-bf55-432996628b80\") " pod="openshift-network-operator/iptables-alerter-dpq9v"
Sep 08 20:25:29.044371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.043383    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ae6a91db-aacf-4fea-8169-a33821987ca7-socket-dir\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:29.045210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.041907    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-cnibin\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:29.045210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.043405    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ddd3b60-46d5-4acf-a364-110c0486e506-host-var-lib-cni-bin\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.045210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.043470    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-log-socket\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.045210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.043492    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.045210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.043624    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de02ff1a-312a-4069-b121-dff1aa3b9354-tmp-dir\") pod \"node-resolver-cljhb\" (UID: \"de02ff1a-312a-4069-b121-dff1aa3b9354\") " pod="openshift-dns/node-resolver-cljhb"
Sep 08 20:25:29.045210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.043687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/daf93c03-ec62-42a5-90e3-ae79820e863f-run-ovn\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.045210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.043805    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/daf93c03-ec62-42a5-90e3-ae79820e863f-env-overrides\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.045210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.044117    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ddd3b60-46d5-4acf-a364-110c0486e506-multus-daemon-config\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.045210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.044126    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:29.045210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.044917    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/daf93c03-ec62-42a5-90e3-ae79820e863f-ovn-node-metrics-cert\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.046922 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.046868    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/15838251-a64d-4585-831a-69d5bb59336b-agent-certs\") pod \"konnectivity-agent-g26sh\" (UID: \"15838251-a64d-4585-831a-69d5bb59336b\") " pod="kube-system/konnectivity-agent-g26sh"
Sep 08 20:25:29.054940 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.054894    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xttd6\" (UniqueName: \"kubernetes.io/projected/1ddd3b60-46d5-4acf-a364-110c0486e506-kube-api-access-xttd6\") pod \"multus-bghvs\" (UID: \"1ddd3b60-46d5-4acf-a364-110c0486e506\") " pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.055100 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.054934    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpkk5\" (UniqueName: \"kubernetes.io/projected/fc3088bd-7250-4a7a-bf55-432996628b80-kube-api-access-wpkk5\") pod \"iptables-alerter-dpq9v\" (UID: \"fc3088bd-7250-4a7a-bf55-432996628b80\") " pod="openshift-network-operator/iptables-alerter-dpq9v"
Sep 08 20:25:29.055100 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.055042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbcl6\" (UniqueName: \"kubernetes.io/projected/de02ff1a-312a-4069-b121-dff1aa3b9354-kube-api-access-lbcl6\") pod \"node-resolver-cljhb\" (UID: \"de02ff1a-312a-4069-b121-dff1aa3b9354\") " pod="openshift-dns/node-resolver-cljhb"
Sep 08 20:25:29.055239 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.055125    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjzc6\" (UniqueName: \"kubernetes.io/projected/642122c1-c2a9-4fe3-aade-12ac05cecf17-kube-api-access-bjzc6\") pod \"network-metrics-daemon-vlx4d\" (UID: \"642122c1-c2a9-4fe3-aade-12ac05cecf17\") " pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:29.056692 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.056643    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmjfv\" (UniqueName: \"kubernetes.io/projected/ae6a91db-aacf-4fea-8169-a33821987ca7-kube-api-access-jmjfv\") pod \"aws-ebs-csi-driver-node-bb5tx\" (UID: \"ae6a91db-aacf-4fea-8169-a33821987ca7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:29.056918 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.056889    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8dvl\" (UniqueName: \"kubernetes.io/projected/df4a9ef6-3242-48f7-b518-6a19f8a3aedd-kube-api-access-k8dvl\") pod \"multus-additional-cni-plugins-gnm8w\" (UID: \"df4a9ef6-3242-48f7-b518-6a19f8a3aedd\") " pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:29.057097 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.057069    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kblr2\" (UniqueName: \"kubernetes.io/projected/daf93c03-ec62-42a5-90e3-ae79820e863f-kube-api-access-kblr2\") pod \"ovnkube-node-x29fz\" (UID: \"daf93c03-ec62-42a5-90e3-ae79820e863f\") " pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.134689 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.134642    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gc9sv"
Sep 08 20:25:29.144193 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.144165    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8h7mp"
Sep 08 20:25:29.146335 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:29.146296    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb691d62f_ea1a_43a9_909e_ab03a8bf9435.slice/crio-58a9e6ee45c386b009ff964155d1f987f9fdb142ad4299a369fcd1f7b4983d9d WatchSource:0}: Error finding container 58a9e6ee45c386b009ff964155d1f987f9fdb142ad4299a369fcd1f7b4983d9d: Status 404 returned error can't find the container with id 58a9e6ee45c386b009ff964155d1f987f9fdb142ad4299a369fcd1f7b4983d9d
Sep 08 20:25:29.154630 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.154547    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:29.154795 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:29.154769    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a574ef3_d788_41e3_bdc3_2e2bd3e10621.slice/crio-4e4ec424f3dcc03885efe82231cd5a33179fd5c128b194d434059df0d30410a5 WatchSource:0}: Error finding container 4e4ec424f3dcc03885efe82231cd5a33179fd5c128b194d434059df0d30410a5: Status 404 returned error can't find the container with id 4e4ec424f3dcc03885efe82231cd5a33179fd5c128b194d434059df0d30410a5
Sep 08 20:25:29.163261 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.163234    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cljhb"
Sep 08 20:25:29.165076 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:29.165020    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaf93c03_ec62_42a5_90e3_ae79820e863f.slice/crio-6beb5ddc222f952f962725a51d8bb4a12ca4f10ca9a6c189983fc3cfc0278416 WatchSource:0}: Error finding container 6beb5ddc222f952f962725a51d8bb4a12ca4f10ca9a6c189983fc3cfc0278416: Status 404 returned error can't find the container with id 6beb5ddc222f952f962725a51d8bb4a12ca4f10ca9a6c189983fc3cfc0278416
Sep 08 20:25:29.165633 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.165533    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 20:25:29.172805 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.172513    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gnm8w"
Sep 08 20:25:29.174254 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:29.174227    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde02ff1a_312a_4069_b121_dff1aa3b9354.slice/crio-e3c82c85e6b1510020aee6a1a0ab30a0e5f0982c705fc175037c0ed2d867fda7 WatchSource:0}: Error finding container e3c82c85e6b1510020aee6a1a0ab30a0e5f0982c705fc175037c0ed2d867fda7: Status 404 returned error can't find the container with id e3c82c85e6b1510020aee6a1a0ab30a0e5f0982c705fc175037c0ed2d867fda7
Sep 08 20:25:29.182072 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.182048    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g26sh"
Sep 08 20:25:29.182246 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:29.182214    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf4a9ef6_3242_48f7_b518_6a19f8a3aedd.slice/crio-a635fc4ab166493c5775804673674cf16e74c3ffd4cb3b4cda6669d9f22b50d5 WatchSource:0}: Error finding container a635fc4ab166493c5775804673674cf16e74c3ffd4cb3b4cda6669d9f22b50d5: Status 404 returned error can't find the container with id a635fc4ab166493c5775804673674cf16e74c3ffd4cb3b4cda6669d9f22b50d5
Sep 08 20:25:29.191437 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.191410    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx"
Sep 08 20:25:29.191972 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:29.191939    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15838251_a64d_4585_831a_69d5bb59336b.slice/crio-63609df2643b0773089ca8eccd44eb575afb2e5a62ad6bc1071b3f611f1a972d WatchSource:0}: Error finding container 63609df2643b0773089ca8eccd44eb575afb2e5a62ad6bc1071b3f611f1a972d: Status 404 returned error can't find the container with id 63609df2643b0773089ca8eccd44eb575afb2e5a62ad6bc1071b3f611f1a972d
Sep 08 20:25:29.199700 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.199675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bghvs"
Sep 08 20:25:29.202020 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:29.201996    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae6a91db_aacf_4fea_8169_a33821987ca7.slice/crio-924f21ea0a14d19d75134a8dbdcf653dae04307c50264046101309e8a65e29ea WatchSource:0}: Error finding container 924f21ea0a14d19d75134a8dbdcf653dae04307c50264046101309e8a65e29ea: Status 404 returned error can't find the container with id 924f21ea0a14d19d75134a8dbdcf653dae04307c50264046101309e8a65e29ea
Sep 08 20:25:29.208709 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.208686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dpq9v"
Sep 08 20:25:29.210065 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:29.210010    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ddd3b60_46d5_4acf_a364_110c0486e506.slice/crio-4a6e2ace1a98b4b610e84cc546e431cf32475b1f2692e3e3652d2e5885968e6f WatchSource:0}: Error finding container 4a6e2ace1a98b4b610e84cc546e431cf32475b1f2692e3e3652d2e5885968e6f: Status 404 returned error can't find the container with id 4a6e2ace1a98b4b610e84cc546e431cf32475b1f2692e3e3652d2e5885968e6f
Sep 08 20:25:29.219243 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:25:29.219060    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc3088bd_7250_4a7a_bf55_432996628b80.slice/crio-49f102aff6d9977f55e488d45a31b2732a701d7f02b6171a76d4843f256deabf WatchSource:0}: Error finding container 49f102aff6d9977f55e488d45a31b2732a701d7f02b6171a76d4843f256deabf: Status 404 returned error can't find the container with id 49f102aff6d9977f55e488d45a31b2732a701d7f02b6171a76d4843f256deabf
Sep 08 20:25:29.331483 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.331391    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 20:25:29.446306 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.446269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret\") pod \"global-pull-secret-syncer-fkjc4\" (UID: \"a9304897-72a4-4daa-aa6e-9397a440502e\") " pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:29.446462 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.446348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdnvw\" (UniqueName: \"kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw\") pod \"network-check-target-vjhfp\" (UID: \"91b9ef98-2cf6-4b27-97b5-01662e8c4058\") " pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:29.446462 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:29.446445    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:29.446462 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:29.446458    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 20:25:29.446613 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:29.446474    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 20:25:29.446613 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:29.446487    2580 projected.go:196] Error preparing data for projected volume kube-api-access-jdnvw for pod openshift-network-diagnostics/network-check-target-vjhfp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:29.446613 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:29.446519    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret podName:a9304897-72a4-4daa-aa6e-9397a440502e nodeName:}" failed. No retries permitted until 2026-09-08 20:25:30.446499415 +0000 UTC m=+3.993748895 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret") pod "global-pull-secret-syncer-fkjc4" (UID: "a9304897-72a4-4daa-aa6e-9397a440502e") : object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:29.446613 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:29.446539    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw podName:91b9ef98-2cf6-4b27-97b5-01662e8c4058 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:30.446529501 +0000 UTC m=+3.993778964 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jdnvw" (UniqueName: "kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw") pod "network-check-target-vjhfp" (UID: "91b9ef98-2cf6-4b27-97b5-01662e8c4058") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:29.547661 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.547620    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs\") pod \"network-metrics-daemon-vlx4d\" (UID: \"642122c1-c2a9-4fe3-aade-12ac05cecf17\") " pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:29.547822 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:29.547785    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:29.547881 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:29.547870    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs podName:642122c1-c2a9-4fe3-aade-12ac05cecf17 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:30.547847632 +0000 UTC m=+4.095097111 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs") pod "network-metrics-daemon-vlx4d" (UID: "642122c1-c2a9-4fe3-aade-12ac05cecf17") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:29.875564 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.875492    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 20:20:29 +0000 UTC" deadline="2028-02-24 08:06:20.391592821 +0000 UTC"
Sep 08 20:25:29.875564 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.875559    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12803h40m50.516037483s"
Sep 08 20:25:29.918804 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.918105    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:29.918804 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:29.918267    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:29.918804 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.918104    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:29.918804 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:29.918750    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:25:29.933797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.933762    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cljhb" event={"ID":"de02ff1a-312a-4069-b121-dff1aa3b9354","Type":"ContainerStarted","Data":"e3c82c85e6b1510020aee6a1a0ab30a0e5f0982c705fc175037c0ed2d867fda7"}
Sep 08 20:25:29.937188 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.937079    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz" event={"ID":"daf93c03-ec62-42a5-90e3-ae79820e863f","Type":"ContainerStarted","Data":"6beb5ddc222f952f962725a51d8bb4a12ca4f10ca9a6c189983fc3cfc0278416"}
Sep 08 20:25:29.940562 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.940531    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8h7mp" event={"ID":"5a574ef3-d788-41e3-bdc3-2e2bd3e10621","Type":"ContainerStarted","Data":"4e4ec424f3dcc03885efe82231cd5a33179fd5c128b194d434059df0d30410a5"}
Sep 08 20:25:29.943980 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.943949    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gc9sv" event={"ID":"b691d62f-ea1a-43a9-909e-ab03a8bf9435","Type":"ContainerStarted","Data":"58a9e6ee45c386b009ff964155d1f987f9fdb142ad4299a369fcd1f7b4983d9d"}
Sep 08 20:25:29.945851 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.945790    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bghvs" event={"ID":"1ddd3b60-46d5-4acf-a364-110c0486e506","Type":"ContainerStarted","Data":"4a6e2ace1a98b4b610e84cc546e431cf32475b1f2692e3e3652d2e5885968e6f"}
Sep 08 20:25:29.947720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.947647    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx" event={"ID":"ae6a91db-aacf-4fea-8169-a33821987ca7","Type":"ContainerStarted","Data":"924f21ea0a14d19d75134a8dbdcf653dae04307c50264046101309e8a65e29ea"}
Sep 08 20:25:29.949551 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.949526    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dpq9v" event={"ID":"fc3088bd-7250-4a7a-bf55-432996628b80","Type":"ContainerStarted","Data":"49f102aff6d9977f55e488d45a31b2732a701d7f02b6171a76d4843f256deabf"}
Sep 08 20:25:29.951069 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.951046    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g26sh" event={"ID":"15838251-a64d-4585-831a-69d5bb59336b","Type":"ContainerStarted","Data":"63609df2643b0773089ca8eccd44eb575afb2e5a62ad6bc1071b3f611f1a972d"}
Sep 08 20:25:29.953239 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:29.953213    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnm8w" event={"ID":"df4a9ef6-3242-48f7-b518-6a19f8a3aedd","Type":"ContainerStarted","Data":"a635fc4ab166493c5775804673674cf16e74c3ffd4cb3b4cda6669d9f22b50d5"}
Sep 08 20:25:30.456297 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:30.456251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdnvw\" (UniqueName: \"kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw\") pod \"network-check-target-vjhfp\" (UID: \"91b9ef98-2cf6-4b27-97b5-01662e8c4058\") " pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:30.456492 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:30.456325    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret\") pod \"global-pull-secret-syncer-fkjc4\" (UID: \"a9304897-72a4-4daa-aa6e-9397a440502e\") " pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:30.456492 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:30.456432    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:30.456627 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:30.456496    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret podName:a9304897-72a4-4daa-aa6e-9397a440502e nodeName:}" failed. No retries permitted until 2026-09-08 20:25:32.456482081 +0000 UTC m=+6.003731542 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret") pod "global-pull-secret-syncer-fkjc4" (UID: "a9304897-72a4-4daa-aa6e-9397a440502e") : object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:30.456896 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:30.456877    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 20:25:30.456960 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:30.456908    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 20:25:30.456960 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:30.456923    2580 projected.go:196] Error preparing data for projected volume kube-api-access-jdnvw for pod openshift-network-diagnostics/network-check-target-vjhfp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:30.457048 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:30.456965    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw podName:91b9ef98-2cf6-4b27-97b5-01662e8c4058 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:32.456954269 +0000 UTC m=+6.004203730 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jdnvw" (UniqueName: "kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw") pod "network-check-target-vjhfp" (UID: "91b9ef98-2cf6-4b27-97b5-01662e8c4058") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:30.557620 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:30.557036    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs\") pod \"network-metrics-daemon-vlx4d\" (UID: \"642122c1-c2a9-4fe3-aade-12ac05cecf17\") " pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:30.557620 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:30.557193    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:30.557620 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:30.557280    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs podName:642122c1-c2a9-4fe3-aade-12ac05cecf17 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:32.557266101 +0000 UTC m=+6.104515561 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs") pod "network-metrics-daemon-vlx4d" (UID: "642122c1-c2a9-4fe3-aade-12ac05cecf17") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:30.876473 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:30.876351    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 20:20:29 +0000 UTC" deadline="2028-05-15 02:42:36.002109659 +0000 UTC"
Sep 08 20:25:30.876473 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:30.876394    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14742h17m5.125720258s"
Sep 08 20:25:30.918289 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:30.917815    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:30.918289 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:30.917925    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:31.918587 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:31.917775    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:31.918587 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:31.917919    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:25:31.918587 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:31.918346    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:31.918587 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:31.918467    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:31.967670 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:31.967626    2580 generic.go:356] "Generic (PLEG): container finished" podID="1e47a49cba2d858e2e9885f9d721c4db" containerID="efbc8030367df213a518b6c891ab53a439638996cbc2f5379acc09e0acc5508f" exitCode=0
Sep 08 20:25:31.967866 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:31.967751    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal" event={"ID":"1e47a49cba2d858e2e9885f9d721c4db","Type":"ContainerDied","Data":"efbc8030367df213a518b6c891ab53a439638996cbc2f5379acc09e0acc5508f"}
Sep 08 20:25:31.976184 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:31.976148    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-142.ec2.internal" event={"ID":"4dade65b3f03b9eb8dc1b57ffacbb7fd","Type":"ContainerStarted","Data":"5525aefdf667b0f7d2008344e666a562d5c08a3a8d9a3296f42e84a97c5f9843"}
Sep 08 20:25:32.476206 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:32.476150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdnvw\" (UniqueName: \"kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw\") pod \"network-check-target-vjhfp\" (UID: \"91b9ef98-2cf6-4b27-97b5-01662e8c4058\") " pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:32.476409 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:32.476214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret\") pod \"global-pull-secret-syncer-fkjc4\" (UID: \"a9304897-72a4-4daa-aa6e-9397a440502e\") " pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:32.476409 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:32.476324    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:32.476409 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:32.476334    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 20:25:32.476409 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:32.476354    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 20:25:32.476409 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:32.476367    2580 projected.go:196] Error preparing data for projected volume kube-api-access-jdnvw for pod openshift-network-diagnostics/network-check-target-vjhfp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:32.476409 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:32.476388    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret podName:a9304897-72a4-4daa-aa6e-9397a440502e nodeName:}" failed. No retries permitted until 2026-09-08 20:25:36.476368677 +0000 UTC m=+10.023618141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret") pod "global-pull-secret-syncer-fkjc4" (UID: "a9304897-72a4-4daa-aa6e-9397a440502e") : object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:32.476409 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:32.476406    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw podName:91b9ef98-2cf6-4b27-97b5-01662e8c4058 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:36.476395295 +0000 UTC m=+10.023644774 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jdnvw" (UniqueName: "kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw") pod "network-check-target-vjhfp" (UID: "91b9ef98-2cf6-4b27-97b5-01662e8c4058") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:32.577046 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:32.576989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs\") pod \"network-metrics-daemon-vlx4d\" (UID: \"642122c1-c2a9-4fe3-aade-12ac05cecf17\") " pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:32.577229 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:32.577185    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:32.577305 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:32.577240    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs podName:642122c1-c2a9-4fe3-aade-12ac05cecf17 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:36.577225776 +0000 UTC m=+10.124475237 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs") pod "network-metrics-daemon-vlx4d" (UID: "642122c1-c2a9-4fe3-aade-12ac05cecf17") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:32.920633 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:32.920547    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:32.921136 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:32.920695    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:33.918294 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:33.918257    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:33.918498 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:33.918411    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:25:33.918883 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:33.918860    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:33.919011 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:33.918992    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:34.918245 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:34.918206    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:34.918746 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:34.918363    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:35.917770 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:35.917683    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:35.917948 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:35.917861    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:25:35.918100 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:35.918069    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:35.918218 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:35.918199    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:36.516182 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:36.516145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdnvw\" (UniqueName: \"kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw\") pod \"network-check-target-vjhfp\" (UID: \"91b9ef98-2cf6-4b27-97b5-01662e8c4058\") " pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:36.516825 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:36.516205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret\") pod \"global-pull-secret-syncer-fkjc4\" (UID: \"a9304897-72a4-4daa-aa6e-9397a440502e\") " pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:36.516825 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:36.516353    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 20:25:36.516825 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:36.516386    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 20:25:36.516825 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:36.516399    2580 projected.go:196] Error preparing data for projected volume kube-api-access-jdnvw for pod openshift-network-diagnostics/network-check-target-vjhfp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:36.516825 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:36.516362    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:36.516825 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:36.516467    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw podName:91b9ef98-2cf6-4b27-97b5-01662e8c4058 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:44.516445989 +0000 UTC m=+18.063695475 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jdnvw" (UniqueName: "kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw") pod "network-check-target-vjhfp" (UID: "91b9ef98-2cf6-4b27-97b5-01662e8c4058") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:36.516825 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:36.516511    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret podName:a9304897-72a4-4daa-aa6e-9397a440502e nodeName:}" failed. No retries permitted until 2026-09-08 20:25:44.516491275 +0000 UTC m=+18.063740735 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret") pod "global-pull-secret-syncer-fkjc4" (UID: "a9304897-72a4-4daa-aa6e-9397a440502e") : object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:36.616994 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:36.616946    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs\") pod \"network-metrics-daemon-vlx4d\" (UID: \"642122c1-c2a9-4fe3-aade-12ac05cecf17\") " pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:36.617177 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:36.617136    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:36.617263 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:36.617221    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs podName:642122c1-c2a9-4fe3-aade-12ac05cecf17 nodeName:}" failed. No retries permitted until 2026-09-08 20:25:44.617197954 +0000 UTC m=+18.164447418 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs") pod "network-metrics-daemon-vlx4d" (UID: "642122c1-c2a9-4fe3-aade-12ac05cecf17") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:36.919333 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:36.919245    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:36.919503 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:36.919383    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:37.918290 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:37.918253    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:37.918740 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:37.918382    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:25:37.918798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:37.918781    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:37.918979 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:37.918895    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:38.918556 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:38.918050    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:38.918556 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:38.918193    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:39.917767 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:39.917729    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:39.917951 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:39.917742    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:39.917951 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:39.917887    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:39.918072 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:39.917995    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:25:40.918154 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:40.918074    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:40.918605 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:40.918189    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:41.918229 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:41.918195    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:41.918641 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:41.918203    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:41.918641 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:41.918321    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:25:41.918641 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:41.918430    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:42.918172 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:42.918130    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:42.918357 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:42.918270    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:43.918193 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:43.918164    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:43.918401 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:43.918172    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:43.918401 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:43.918279    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:25:43.918401 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:43.918364    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:44.577909 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:44.577821    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret\") pod \"global-pull-secret-syncer-fkjc4\" (UID: \"a9304897-72a4-4daa-aa6e-9397a440502e\") " pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:44.578080 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:44.577914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdnvw\" (UniqueName: \"kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw\") pod \"network-check-target-vjhfp\" (UID: \"91b9ef98-2cf6-4b27-97b5-01662e8c4058\") " pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:44.578080 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:44.577998    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:44.578080 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:44.578039    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 20:25:44.578080 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:44.578058    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 20:25:44.578080 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:44.578069    2580 projected.go:196] Error preparing data for projected volume kube-api-access-jdnvw for pod openshift-network-diagnostics/network-check-target-vjhfp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:44.578080 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:44.578082    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret podName:a9304897-72a4-4daa-aa6e-9397a440502e nodeName:}" failed. No retries permitted until 2026-09-08 20:26:00.578052046 +0000 UTC m=+34.125301521 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret") pod "global-pull-secret-syncer-fkjc4" (UID: "a9304897-72a4-4daa-aa6e-9397a440502e") : object "kube-system"/"original-pull-secret" not registered
Sep 08 20:25:44.578273 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:44.578104    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw podName:91b9ef98-2cf6-4b27-97b5-01662e8c4058 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:00.578093705 +0000 UTC m=+34.125343164 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jdnvw" (UniqueName: "kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw") pod "network-check-target-vjhfp" (UID: "91b9ef98-2cf6-4b27-97b5-01662e8c4058") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:25:44.678621 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:44.678581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs\") pod \"network-metrics-daemon-vlx4d\" (UID: \"642122c1-c2a9-4fe3-aade-12ac05cecf17\") " pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:44.678797 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:44.678763    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:44.678854 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:44.678845    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs podName:642122c1-c2a9-4fe3-aade-12ac05cecf17 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:00.678823226 +0000 UTC m=+34.226072691 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs") pod "network-metrics-daemon-vlx4d" (UID: "642122c1-c2a9-4fe3-aade-12ac05cecf17") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:25:44.917565 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:44.917476    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:44.917723 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:44.917609    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:45.917766 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:45.917727    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:45.918252 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:45.917738    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:45.918252 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:45.917859    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:25:45.918252 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:45.917918    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:46.918382 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:46.918345    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:46.918832 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:46.918457    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:47.917638 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:47.917596    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:47.917797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:47.917604    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:47.917797 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:47.917739    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:25:47.917890 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:47.917800    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:48.918209 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:48.917953    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:48.918891 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:48.918247    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:49.018482 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.018440    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g26sh" event={"ID":"15838251-a64d-4585-831a-69d5bb59336b","Type":"ContainerStarted","Data":"cd8fc85a94d5319448ef1f3cf7c23686436d4fa9ddc3a43d984c1d83665f3ee7"}
Sep 08 20:25:49.020275 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.020249    2580 generic.go:356] "Generic (PLEG): container finished" podID="df4a9ef6-3242-48f7-b518-6a19f8a3aedd" containerID="198004a21fe26e6621e9fb38ba428a21761d865d096744c763c580f534a8f1fb" exitCode=0
Sep 08 20:25:49.020403 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.020335    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnm8w" event={"ID":"df4a9ef6-3242-48f7-b518-6a19f8a3aedd","Type":"ContainerDied","Data":"198004a21fe26e6621e9fb38ba428a21761d865d096744c763c580f534a8f1fb"}
Sep 08 20:25:49.022211 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.022080    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cljhb" event={"ID":"de02ff1a-312a-4069-b121-dff1aa3b9354","Type":"ContainerStarted","Data":"edbc280c08b42f29357c933b93f61455df430fdf980634838a78f36de3896caa"}
Sep 08 20:25:49.025555 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.025533    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz" event={"ID":"daf93c03-ec62-42a5-90e3-ae79820e863f","Type":"ContainerStarted","Data":"3b3a4164d344ecadf39c00172ff5866a0884f98abc105aedc2f8657118816355"}
Sep 08 20:25:49.025654 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.025602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz" event={"ID":"daf93c03-ec62-42a5-90e3-ae79820e863f","Type":"ContainerStarted","Data":"a0ab8f5c1717c7c55248d8679e8c6666101d8dc41ef832c9dce272048ff529e8"}
Sep 08 20:25:49.025713 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.025655    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz" event={"ID":"daf93c03-ec62-42a5-90e3-ae79820e863f","Type":"ContainerStarted","Data":"bec5f75acf8519eb9ed833fdb0096de9d213530b7bdaca6e0e9995976e6d0abd"}
Sep 08 20:25:49.025713 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.025700    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz" event={"ID":"daf93c03-ec62-42a5-90e3-ae79820e863f","Type":"ContainerStarted","Data":"dcc9a68c920f2a89de244f5d4f8726aa54ade6856b54e5014f669af7ac7b2c38"}
Sep 08 20:25:49.025807 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.025735    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz" event={"ID":"daf93c03-ec62-42a5-90e3-ae79820e863f","Type":"ContainerStarted","Data":"196c447146ff46ac8a8b81302ba8193adb1bfb1537329b7ec67384fca7bdf8fc"}
Sep 08 20:25:49.025807 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.025781    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz" event={"ID":"daf93c03-ec62-42a5-90e3-ae79820e863f","Type":"ContainerStarted","Data":"4bab34829f569a4cb42d1c58df868e19bb86fb0ab8b628e90b8890448d6c323b"}
Sep 08 20:25:49.027205 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.027185    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8h7mp" event={"ID":"5a574ef3-d788-41e3-bdc3-2e2bd3e10621","Type":"ContainerStarted","Data":"901c191c5d5371e2fd4cb9544b3417c098b4091ec91519b52b4aa11cda02915e"}
Sep 08 20:25:49.028756 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.028714    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gc9sv" event={"ID":"b691d62f-ea1a-43a9-909e-ab03a8bf9435","Type":"ContainerStarted","Data":"3b0863fc54120f76a8908a0c01d3ea251d3af5f060de07ea6990b8290a6baacb"}
Sep 08 20:25:49.030684 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.030661    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal" event={"ID":"1e47a49cba2d858e2e9885f9d721c4db","Type":"ContainerStarted","Data":"aeef3f8f5abfe0167deaadb6f22c4b4feb5dd4555805d46bb095ea9943448ddc"}
Sep 08 20:25:49.032301 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.032280    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bghvs" event={"ID":"1ddd3b60-46d5-4acf-a364-110c0486e506","Type":"ContainerStarted","Data":"2023725e5bf96ce54929f81fb6e5f2b0c9aa67df8d54f6c915b734504fc33397"}
Sep 08 20:25:49.033834 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.033814    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx" event={"ID":"ae6a91db-aacf-4fea-8169-a33821987ca7","Type":"ContainerStarted","Data":"6201a2a04834ca197cb34bac4a8d546ea7be3027963fd4ce34b398bb528d045d"}
Sep 08 20:25:49.034607 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.034563    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g26sh" podStartSLOduration=3.185064601 podStartE2EDuration="22.03455133s" totalImagesPullingTime="18.849486729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:49.034097393 +0000 UTC m=+22.581346875" watchObservedRunningTime="2026-09-08 20:25:49.03455133 +0000 UTC m=+22.581800811"
Sep 08 20:25:49.034711 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.034657    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-142.ec2.internal" podStartSLOduration=21.034651216 podStartE2EDuration="21.034651216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:32.000875736 +0000 UTC m=+5.548125247" watchObservedRunningTime="2026-09-08 20:25:49.034651216 +0000 UTC m=+22.581900700"
Sep 08 20:25:49.053697 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.053640    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-142.ec2.internal" podStartSLOduration=21.053624366 podStartE2EDuration="21.053624366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:49.05316254 +0000 UTC m=+22.600412022" watchObservedRunningTime="2026-09-08 20:25:49.053624366 +0000 UTC m=+22.600873846"
Sep 08 20:25:49.067952 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.067900    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gc9sv" podStartSLOduration=3.172954729 podStartE2EDuration="22.067881877s" totalImagesPullingTime="18.894927148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:49.0675258 +0000 UTC m=+22.614775270" watchObservedRunningTime="2026-09-08 20:25:49.067881877 +0000 UTC m=+22.615131383"
Sep 08 20:25:49.088369 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.088309    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8h7mp" podStartSLOduration=3.173324935 podStartE2EDuration="22.088291271s" totalImagesPullingTime="18.914966336s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:49.087796087 +0000 UTC m=+22.635045567" watchObservedRunningTime="2026-09-08 20:25:49.088291271 +0000 UTC m=+22.635540754"
Sep 08 20:25:49.109931 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.109892    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:49.160109 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.158207    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cljhb" podStartSLOduration=3.290894623 podStartE2EDuration="22.158185923s" totalImagesPullingTime="18.8672913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:49.157273371 +0000 UTC m=+22.704522851" watchObservedRunningTime="2026-09-08 20:25:49.158185923 +0000 UTC m=+22.705435405"
Sep 08 20:25:49.160109 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.158585    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bghvs" podStartSLOduration=3.246660491 podStartE2EDuration="22.158568446s" totalImagesPullingTime="18.911907955s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:49.132472092 +0000 UTC m=+22.679721574" watchObservedRunningTime="2026-09-08 20:25:49.158568446 +0000 UTC m=+22.705817928"
Sep 08 20:25:49.231338 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.231261    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:49.331532 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.331496    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:49.620434 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.620175    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 08 20:25:49.883224 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.883128    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T20:25:49.620431261Z","UUID":"fc6dfc95-b417-4735-ac96-072c01ce072a","Handler":null,"Name":"","Endpoint":""}
Sep 08 20:25:49.884833 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.884807    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 08 20:25:49.884978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.884846    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 08 20:25:49.917316 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.917289    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:49.917316 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:49.917309    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:49.917531 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:49.917409    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:25:49.917636 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:49.917530    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:50.004605 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:50.004565    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:50.038190 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:50.038150    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dpq9v" event={"ID":"fc3088bd-7250-4a7a-bf55-432996628b80","Type":"ContainerStarted","Data":"15284acef35f22fcab9d64f4bc1199044f858b8c2e12a032b20545143cb016a8"}
Sep 08 20:25:50.040310 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:50.040213    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx" event={"ID":"ae6a91db-aacf-4fea-8169-a33821987ca7","Type":"ContainerStarted","Data":"235f7b78ab5d7b8e08920409a801b30b3bb6fac222e82467788917f4a62fab61"}
Sep 08 20:25:50.054718 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:50.054667    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dpq9v" podStartSLOduration=4.23242775 podStartE2EDuration="23.054650415s" totalImagesPullingTime="18.822222665s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:50.054097607 +0000 UTC m=+23.601347087" watchObservedRunningTime="2026-09-08 20:25:50.054650415 +0000 UTC m=+23.601899896"
Sep 08 20:25:50.104339 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:50.104298    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:50.396916 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:50.396873    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:50.493793 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:50.493749    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:50.918381 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:50.918309    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:50.918522 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:50.918413    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:51.045454 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:51.045417    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz" event={"ID":"daf93c03-ec62-42a5-90e3-ae79820e863f","Type":"ContainerStarted","Data":"9cd4096475d44f2fd380c96b06f3e6e4f78b41be6d1f441e079db4e5e129bad1"}
Sep 08 20:25:51.047322 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:51.047297    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx" event={"ID":"ae6a91db-aacf-4fea-8169-a33821987ca7","Type":"ContainerStarted","Data":"dd138f3320d9d7cf84724c627376839db5a07a6f44636cb2361a02bc6429966b"}
Sep 08 20:25:51.066337 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:51.066292    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bb5tx" podStartSLOduration=3.088597052 podStartE2EDuration="24.066277447s" totalImagesPullingTime="20.977680395s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:51.065776676 +0000 UTC m=+24.613026156" watchObservedRunningTime="2026-09-08 20:25:51.066277447 +0000 UTC m=+24.613526927"
Sep 08 20:25:51.204917 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:51.204835    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:51.595274 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:51.595198    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:51.661309 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:51.661271    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:51.918308 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:51.918280    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:51.918496 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:51.918280    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:51.918496 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:51.918414    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:51.918496 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:51.918481    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:25:52.103290 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:52.103234    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:52.170847 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:52.170766    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:52.597738 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:52.597662    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g26sh"
Sep 08 20:25:52.598396 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:52.598377    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g26sh"
Sep 08 20:25:52.789331 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:52.789285    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:52.888825 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:52.888775    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:52.917887 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:52.917852    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:52.918081 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:52.918000    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:53.918246 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:53.918132    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:53.918246 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:53.918165    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:53.918976 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:53.918269    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:25:53.918976 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:53.918436    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:54.055425 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:54.055346    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnm8w" event={"ID":"df4a9ef6-3242-48f7-b518-6a19f8a3aedd","Type":"ContainerStarted","Data":"37a112f519e9e642080545201afac44fc9913e76f3ab08cac03597770e1ad6eb"}
Sep 08 20:25:54.064113 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:54.064081    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz" event={"ID":"daf93c03-ec62-42a5-90e3-ae79820e863f","Type":"ContainerStarted","Data":"88538843e2f2e3925abdafaf6a7c5e5ffe3d114c11d6e7e0159198c2bfca24d4"}
Sep 08 20:25:54.064337 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:54.064313    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:54.064423 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:54.064359    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:54.079346 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:54.079320    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:54.103620 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:54.103565    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz" podStartSLOduration=8.175459614 podStartE2EDuration="27.103547621s" totalImagesPullingTime="18.928088007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:25:54.103096159 +0000 UTC m=+27.650345640" watchObservedRunningTime="2026-09-08 20:25:54.103547621 +0000 UTC m=+27.650797101"
Sep 08 20:25:54.917727 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:54.917691    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:54.917871 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:54.917821    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:55.072002 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:55.069888    2580 generic.go:356] "Generic (PLEG): container finished" podID="df4a9ef6-3242-48f7-b518-6a19f8a3aedd" containerID="37a112f519e9e642080545201afac44fc9913e76f3ab08cac03597770e1ad6eb" exitCode=0
Sep 08 20:25:55.072002 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:55.071738    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnm8w" event={"ID":"df4a9ef6-3242-48f7-b518-6a19f8a3aedd","Type":"ContainerDied","Data":"37a112f519e9e642080545201afac44fc9913e76f3ab08cac03597770e1ad6eb"}
Sep 08 20:25:55.072002 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:55.071784    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnm8w" event={"ID":"df4a9ef6-3242-48f7-b518-6a19f8a3aedd","Type":"ContainerStarted","Data":"62711cf309f35e958eef59f1a0dd5525cf89e8b2831a099afd3c36596d46af35"}
Sep 08 20:25:55.072002 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:55.071897    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:55.090325 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:55.090293    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:25:55.738282 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:55.738100    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vjhfp"]
Sep 08 20:25:55.738441 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:55.738344    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:55.738483 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:55.738432    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:55.741532 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:55.741498    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fkjc4"]
Sep 08 20:25:55.741649 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:55.741613    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:55.741764 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:55.741709    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:25:55.742246 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:55.742223    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vlx4d"]
Sep 08 20:25:55.742325 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:55.742318    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:55.742613 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:55.742402    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:56.074788 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:56.074757    2580 generic.go:356] "Generic (PLEG): container finished" podID="df4a9ef6-3242-48f7-b518-6a19f8a3aedd" containerID="62711cf309f35e958eef59f1a0dd5525cf89e8b2831a099afd3c36596d46af35" exitCode=0
Sep 08 20:25:56.075334 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:56.074843    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnm8w" event={"ID":"df4a9ef6-3242-48f7-b518-6a19f8a3aedd","Type":"ContainerDied","Data":"62711cf309f35e958eef59f1a0dd5525cf89e8b2831a099afd3c36596d46af35"}
Sep 08 20:25:56.931173 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:56.931137    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:56.991358 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:56.991274    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:57.079890 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:57.079854    2580 generic.go:356] "Generic (PLEG): container finished" podID="df4a9ef6-3242-48f7-b518-6a19f8a3aedd" containerID="7d4f9303070c226b0d989cb74e28bb224ebbbaf11c94f80c4d9abe658e12be30" exitCode=0
Sep 08 20:25:57.080388 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:57.079941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnm8w" event={"ID":"df4a9ef6-3242-48f7-b518-6a19f8a3aedd","Type":"ContainerDied","Data":"7d4f9303070c226b0d989cb74e28bb224ebbbaf11c94f80c4d9abe658e12be30"}
Sep 08 20:25:57.763063 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:57.763024    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g26sh"
Sep 08 20:25:57.763251 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:57.763191    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 08 20:25:57.763776 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:57.763756    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g26sh"
Sep 08 20:25:57.821292 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:57.821252    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:25:57.917971 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:57.917923    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:57.917971 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:57.917953    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:57.918170 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:57.917953    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:57.918170 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:57.918081    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:57.918278 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:57.918197    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:57.918278 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:57.918270    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:25:59.917665 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:59.917630    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:25:59.918091 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:59.917630    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:25:59.918091 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:59.917754    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vlx4d" podUID="642122c1-c2a9-4fe3-aade-12ac05cecf17"
Sep 08 20:25:59.918091 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:25:59.917630    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:25:59.918091 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:59.917814    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vjhfp" podUID="91b9ef98-2cf6-4b27-97b5-01662e8c4058"
Sep 08 20:25:59.918091 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:25:59.917897    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fkjc4" podUID="a9304897-72a4-4daa-aa6e-9397a440502e"
Sep 08 20:26:00.597095 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.597064    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdnvw\" (UniqueName: \"kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw\") pod \"network-check-target-vjhfp\" (UID: \"91b9ef98-2cf6-4b27-97b5-01662e8c4058\") " pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:26:00.597337 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.597107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret\") pod \"global-pull-secret-syncer-fkjc4\" (UID: \"a9304897-72a4-4daa-aa6e-9397a440502e\") " pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:26:00.597337 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:00.597212    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 20:26:00.597337 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:00.597224    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 20:26:00.597337 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:00.597244    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 20:26:00.597337 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:00.597254    2580 projected.go:196] Error preparing data for projected volume kube-api-access-jdnvw for pod openshift-network-diagnostics/network-check-target-vjhfp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:26:00.597337 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:00.597265    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret podName:a9304897-72a4-4daa-aa6e-9397a440502e nodeName:}" failed. No retries permitted until 2026-09-08 20:26:32.597250543 +0000 UTC m=+66.144500010 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret") pod "global-pull-secret-syncer-fkjc4" (UID: "a9304897-72a4-4daa-aa6e-9397a440502e") : object "kube-system"/"original-pull-secret" not registered
Sep 08 20:26:00.597337 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:00.597292    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw podName:91b9ef98-2cf6-4b27-97b5-01662e8c4058 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:32.597281165 +0000 UTC m=+66.144530625 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jdnvw" (UniqueName: "kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw") pod "network-check-target-vjhfp" (UID: "91b9ef98-2cf6-4b27-97b5-01662e8c4058") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 20:26:00.698037 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.697999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs\") pod \"network-metrics-daemon-vlx4d\" (UID: \"642122c1-c2a9-4fe3-aade-12ac05cecf17\") " pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:26:00.698215 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:00.698109    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:26:00.698215 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:00.698161    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs podName:642122c1-c2a9-4fe3-aade-12ac05cecf17 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:32.698147363 +0000 UTC m=+66.245396823 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs") pod "network-metrics-daemon-vlx4d" (UID: "642122c1-c2a9-4fe3-aade-12ac05cecf17") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 20:26:00.798497 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.798468    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-142.ec2.internal" event="NodeReady"
Sep 08 20:26:00.798671 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.798588    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 08 20:26:00.844479 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.844444    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-75k8c"]
Sep 08 20:26:00.866588 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.866263    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ck7k5"]
Sep 08 20:26:00.866588 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.866525    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:00.868888 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.868867    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 08 20:26:00.869064 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.868867    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vbwpr\""
Sep 08 20:26:00.869064 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.869039    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 08 20:26:00.877310 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.877281    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ck7k5"]
Sep 08 20:26:00.877437 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.877325    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-75k8c"]
Sep 08 20:26:00.877437 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.877395    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ck7k5"
Sep 08 20:26:00.879756 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.879735    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 08 20:26:00.879865 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.879795    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-t62jz\""
Sep 08 20:26:00.879865 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.879735    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 08 20:26:00.879988 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.879746    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 08 20:26:00.966004 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.965952    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:01.000019 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:00.999982    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34b69620-f3c4-471b-85d5-887e22e90910-tmp-dir\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:01.000195 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.000031    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b838c12-7a7d-432e-ad79-d05e1104643f-cert\") pod \"ingress-canary-ck7k5\" (UID: \"0b838c12-7a7d-432e-ad79-d05e1104643f\") " pod="openshift-ingress-canary/ingress-canary-ck7k5"
Sep 08 20:26:01.000195 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.000115    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c6wr\" (UniqueName: \"kubernetes.io/projected/0b838c12-7a7d-432e-ad79-d05e1104643f-kube-api-access-5c6wr\") pod \"ingress-canary-ck7k5\" (UID: \"0b838c12-7a7d-432e-ad79-d05e1104643f\") " pod="openshift-ingress-canary/ingress-canary-ck7k5"
Sep 08 20:26:01.000195 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.000168    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr9xx\" (UniqueName: \"kubernetes.io/projected/34b69620-f3c4-471b-85d5-887e22e90910-kube-api-access-vr9xx\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:01.000330 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.000197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34b69620-f3c4-471b-85d5-887e22e90910-config-volume\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:01.000330 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.000213    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:01.028435 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.028394    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:01.101412 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.101373    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34b69620-f3c4-471b-85d5-887e22e90910-tmp-dir\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:01.101412 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.101417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b838c12-7a7d-432e-ad79-d05e1104643f-cert\") pod \"ingress-canary-ck7k5\" (UID: \"0b838c12-7a7d-432e-ad79-d05e1104643f\") " pod="openshift-ingress-canary/ingress-canary-ck7k5"
Sep 08 20:26:01.101655 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.101547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c6wr\" (UniqueName: \"kubernetes.io/projected/0b838c12-7a7d-432e-ad79-d05e1104643f-kube-api-access-5c6wr\") pod \"ingress-canary-ck7k5\" (UID: \"0b838c12-7a7d-432e-ad79-d05e1104643f\") " pod="openshift-ingress-canary/ingress-canary-ck7k5"
Sep 08 20:26:01.101655 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.101613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr9xx\" (UniqueName: \"kubernetes.io/projected/34b69620-f3c4-471b-85d5-887e22e90910-kube-api-access-vr9xx\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:01.101655 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.101641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34b69620-f3c4-471b-85d5-887e22e90910-config-volume\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:01.101814 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.101665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:01.101814 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.101736    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34b69620-f3c4-471b-85d5-887e22e90910-tmp-dir\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:01.101814 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:01.101777    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 20:26:01.101972 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:01.101836    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls podName:34b69620-f3c4-471b-85d5-887e22e90910 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:01.601819855 +0000 UTC m=+35.149069320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls") pod "dns-default-75k8c" (UID: "34b69620-f3c4-471b-85d5-887e22e90910") : secret "dns-default-metrics-tls" not found
Sep 08 20:26:01.102505 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.102481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34b69620-f3c4-471b-85d5-887e22e90910-config-volume\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:01.106332 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.106157    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b838c12-7a7d-432e-ad79-d05e1104643f-cert\") pod \"ingress-canary-ck7k5\" (UID: \"0b838c12-7a7d-432e-ad79-d05e1104643f\") " pod="openshift-ingress-canary/ingress-canary-ck7k5"
Sep 08 20:26:01.109888 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.109866    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr9xx\" (UniqueName: \"kubernetes.io/projected/34b69620-f3c4-471b-85d5-887e22e90910-kube-api-access-vr9xx\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:01.110017 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.109938    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c6wr\" (UniqueName: \"kubernetes.io/projected/0b838c12-7a7d-432e-ad79-d05e1104643f-kube-api-access-5c6wr\") pod \"ingress-canary-ck7k5\" (UID: \"0b838c12-7a7d-432e-ad79-d05e1104643f\") " pod="openshift-ingress-canary/ingress-canary-ck7k5"
Sep 08 20:26:01.188010 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.187975    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ck7k5"
Sep 08 20:26:01.605863 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.605776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:01.606065 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:01.605918    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 20:26:01.606065 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:01.606016    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls podName:34b69620-f3c4-471b-85d5-887e22e90910 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:02.605995455 +0000 UTC m=+36.153244918 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls") pod "dns-default-75k8c" (UID: "34b69620-f3c4-471b-85d5-887e22e90910") : secret "dns-default-metrics-tls" not found
Sep 08 20:26:01.918316 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.918261    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:26:01.918316 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.918320    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:26:01.918631 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.918261    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:26:01.920857 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.920838    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 20:26:01.920999 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.920840    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 20:26:01.921928 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.921911    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 20:26:01.922021 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.921911    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kbhtv\""
Sep 08 20:26:01.922021 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.922003    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2nzg7\""
Sep 08 20:26:01.922021 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:01.922012    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 20:26:02.264185 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:02.264165    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ck7k5"]
Sep 08 20:26:02.267889 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:26:02.267863    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b838c12_7a7d_432e_ad79_d05e1104643f.slice/crio-38786ef8f2d28f5fed6637dcbbdf5a54700b0b41bbb8279e5b7c75890a48bd1f WatchSource:0}: Error finding container 38786ef8f2d28f5fed6637dcbbdf5a54700b0b41bbb8279e5b7c75890a48bd1f: Status 404 returned error can't find the container with id 38786ef8f2d28f5fed6637dcbbdf5a54700b0b41bbb8279e5b7c75890a48bd1f
Sep 08 20:26:02.614359 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:02.614263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:02.614501 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:02.614384    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 20:26:02.614501 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:02.614438    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls podName:34b69620-f3c4-471b-85d5-887e22e90910 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:04.614424023 +0000 UTC m=+38.161673482 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls") pod "dns-default-75k8c" (UID: "34b69620-f3c4-471b-85d5-887e22e90910") : secret "dns-default-metrics-tls" not found
Sep 08 20:26:03.098565 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:03.098527    2580 generic.go:356] "Generic (PLEG): container finished" podID="df4a9ef6-3242-48f7-b518-6a19f8a3aedd" containerID="a0a7e8a8352ba8dff1e4a6cb309aa53944f3c78a513d506eb7e2c2b7830ae3d1" exitCode=0
Sep 08 20:26:03.098748 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:03.098610    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnm8w" event={"ID":"df4a9ef6-3242-48f7-b518-6a19f8a3aedd","Type":"ContainerDied","Data":"a0a7e8a8352ba8dff1e4a6cb309aa53944f3c78a513d506eb7e2c2b7830ae3d1"}
Sep 08 20:26:03.100211 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:03.100187    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ck7k5" event={"ID":"0b838c12-7a7d-432e-ad79-d05e1104643f","Type":"ContainerStarted","Data":"38786ef8f2d28f5fed6637dcbbdf5a54700b0b41bbb8279e5b7c75890a48bd1f"}
Sep 08 20:26:04.105772 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:04.105736    2580 generic.go:356] "Generic (PLEG): container finished" podID="df4a9ef6-3242-48f7-b518-6a19f8a3aedd" containerID="ea2ea0a628fac30459e88da938c6c23521abedf0504f48180dca4e2b4edbac0f" exitCode=0
Sep 08 20:26:04.106311 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:04.105791    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnm8w" event={"ID":"df4a9ef6-3242-48f7-b518-6a19f8a3aedd","Type":"ContainerDied","Data":"ea2ea0a628fac30459e88da938c6c23521abedf0504f48180dca4e2b4edbac0f"}
Sep 08 20:26:04.629050 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:04.629012    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:04.629241 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:04.629160    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 20:26:04.629241 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:04.629236    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls podName:34b69620-f3c4-471b-85d5-887e22e90910 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:08.629219601 +0000 UTC m=+42.176469060 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls") pod "dns-default-75k8c" (UID: "34b69620-f3c4-471b-85d5-887e22e90910") : secret "dns-default-metrics-tls" not found
Sep 08 20:26:05.112994 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:05.112789    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnm8w" event={"ID":"df4a9ef6-3242-48f7-b518-6a19f8a3aedd","Type":"ContainerStarted","Data":"2db3f6c4d0e2a4bbc8f5cfa1b46b51da4162032031c59150d251dc4a7127b3aa"}
Sep 08 20:26:05.135735 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:05.135683    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gnm8w" podStartSLOduration=15.104777255 podStartE2EDuration="38.135663222s" totalImagesPullingTime="23.030885967s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:05.135279223 +0000 UTC m=+38.682528695" watchObservedRunningTime="2026-09-08 20:26:05.135663222 +0000 UTC m=+38.682912705"
Sep 08 20:26:05.255663 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:05.255582    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:05.309813 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:05.309780    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:06.117082 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:06.117049    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ck7k5" event={"ID":"0b838c12-7a7d-432e-ad79-d05e1104643f","Type":"ContainerStarted","Data":"2c02b5bb515ec77f12f0852dc1b1bd08c60d3280c8faffab8fc743634df0bc81"}
Sep 08 20:26:06.390248 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:06.390216    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:06.446020 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:06.445992    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:08.657353 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:08.657315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:08.657727 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:08.657492    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 20:26:08.657727 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:08.657563    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls podName:34b69620-f3c4-471b-85d5-887e22e90910 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:16.657544912 +0000 UTC m=+50.204794395 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls") pod "dns-default-75k8c" (UID: "34b69620-f3c4-471b-85d5-887e22e90910") : secret "dns-default-metrics-tls" not found
Sep 08 20:26:09.619108 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:09.619076    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:09.720079 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:09.720046    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:10.606827 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:10.606794    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:10.666065 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:10.666026    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:14.557789 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:14.557752    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:14.805711 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:14.805678    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:14.866750 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:14.866685    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:16.713216 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:16.713184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:16.713655 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:16.713307    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 20:26:16.713655 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:16.713361    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls podName:34b69620-f3c4-471b-85d5-887e22e90910 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:32.713346355 +0000 UTC m=+66.260595814 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls") pod "dns-default-75k8c" (UID: "34b69620-f3c4-471b-85d5-887e22e90910") : secret "dns-default-metrics-tls" not found
Sep 08 20:26:19.026432 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.026374    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ck7k5" podStartSLOduration=16.259300942 podStartE2EDuration="19.026356868s" totalImagesPullingTime="2.767055926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:06.131806027 +0000 UTC m=+39.679055509" watchObservedRunningTime="2026-09-08 20:26:19.026356868 +0000 UTC m=+52.573606395"
Sep 08 20:26:19.026840 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.026818    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl"]
Sep 08 20:26:19.028660 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.028640    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl"
Sep 08 20:26:19.031311 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.031283    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 08 20:26:19.031311 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.031306    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 08 20:26:19.031484 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.031308    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 08 20:26:19.032231 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.032204    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 08 20:26:19.032335 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.032208    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-gjgtj\""
Sep 08 20:26:19.039242 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.039220    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl"]
Sep 08 20:26:19.129471 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.129441    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d374121e-fdb4-4528-bee6-ffd573438cf1-serving-cert\") pod \"service-ca-operator-859f94c4b-qw8tl\" (UID: \"d374121e-fdb4-4528-bee6-ffd573438cf1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl"
Sep 08 20:26:19.129471 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.129472    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d374121e-fdb4-4528-bee6-ffd573438cf1-config\") pod \"service-ca-operator-859f94c4b-qw8tl\" (UID: \"d374121e-fdb4-4528-bee6-ffd573438cf1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl"
Sep 08 20:26:19.129681 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.129591    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg7nh\" (UniqueName: \"kubernetes.io/projected/d374121e-fdb4-4528-bee6-ffd573438cf1-kube-api-access-cg7nh\") pod \"service-ca-operator-859f94c4b-qw8tl\" (UID: \"d374121e-fdb4-4528-bee6-ffd573438cf1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl"
Sep 08 20:26:19.134420 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.134391    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-glk4q"]
Sep 08 20:26:19.136391 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.136373    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-glk4q"
Sep 08 20:26:19.138724 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.138701    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 08 20:26:19.138830 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.138750    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-56w4w\""
Sep 08 20:26:19.138830 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.138784    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 08 20:26:19.141159 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.141136    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq"]
Sep 08 20:26:19.143833 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.143813    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"]
Sep 08 20:26:19.143926 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.143913    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq"
Sep 08 20:26:19.145328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.145312    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-596c57999-j8bwl"]
Sep 08 20:26:19.145420 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.145407    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:19.146865 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.146848    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 08 20:26:19.147415 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.147394    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.150546 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.150520    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-bpqh8\""
Sep 08 20:26:19.150651 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.150563    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 08 20:26:19.150651 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.150574    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 08 20:26:19.151062 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.151037    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 08 20:26:19.151279 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.151256    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 08 20:26:19.151937 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.151912    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 08 20:26:19.153987 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.152538    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 08 20:26:19.153987 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.152340    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 08 20:26:19.153987 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.153209    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 08 20:26:19.153987 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.153547    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-h8wxl\""
Sep 08 20:26:19.153987 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.153869    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 08 20:26:19.153987 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.153875    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nwnm8\""
Sep 08 20:26:19.158205 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.158188    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 08 20:26:19.160337 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.160319    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-glk4q"]
Sep 08 20:26:19.168190 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.168135    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-596c57999-j8bwl"]
Sep 08 20:26:19.169382 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.169359    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"]
Sep 08 20:26:19.170150 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.170130    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq"]
Sep 08 20:26:19.230021 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.229980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d374121e-fdb4-4528-bee6-ffd573438cf1-serving-cert\") pod \"service-ca-operator-859f94c4b-qw8tl\" (UID: \"d374121e-fdb4-4528-bee6-ffd573438cf1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl"
Sep 08 20:26:19.230021 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d374121e-fdb4-4528-bee6-ffd573438cf1-config\") pod \"service-ca-operator-859f94c4b-qw8tl\" (UID: \"d374121e-fdb4-4528-bee6-ffd573438cf1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl"
Sep 08 20:26:19.230251 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230063    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d2ed3c01-5a4d-475b-84f3-97c7969c52da-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-cl6hs\" (UID: \"d2ed3c01-5a4d-475b-84f3-97c7969c52da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:19.230251 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230091    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fm9m\" (UniqueName: \"kubernetes.io/projected/d2ed3c01-5a4d-475b-84f3-97c7969c52da-kube-api-access-5fm9m\") pod \"cluster-monitoring-operator-6cdcc96965-cl6hs\" (UID: \"d2ed3c01-5a4d-475b-84f3-97c7969c52da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:19.230251 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230136    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2129f9ae-e512-4e49-8f03-ec052fb52046-installation-pull-secrets\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.230251 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230163    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2129f9ae-e512-4e49-8f03-ec052fb52046-trusted-ca\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.230251 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j72p\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-kube-api-access-2j72p\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.230443 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230264    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6kmp\" (UniqueName: \"kubernetes.io/projected/5e7ff65d-5192-457e-aac7-8eef1a56d725-kube-api-access-b6kmp\") pod \"volume-data-source-validator-695c895bd6-glk4q\" (UID: \"5e7ff65d-5192-457e-aac7-8eef1a56d725\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-glk4q"
Sep 08 20:26:19.230443 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230317    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-certificates\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.230443 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230343    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2129f9ae-e512-4e49-8f03-ec052fb52046-image-registry-private-configuration\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.230443 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230368    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2129f9ae-e512-4e49-8f03-ec052fb52046-ca-trust-extracted\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.230606 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg7nh\" (UniqueName: \"kubernetes.io/projected/d374121e-fdb4-4528-bee6-ffd573438cf1-kube-api-access-cg7nh\") pod \"service-ca-operator-859f94c4b-qw8tl\" (UID: \"d374121e-fdb4-4528-bee6-ffd573438cf1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl"
Sep 08 20:26:19.230606 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230496    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cl6hs\" (UID: \"d2ed3c01-5a4d-475b-84f3-97c7969c52da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:19.230606 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230533    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.230606 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230561    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx5mq\" (UniqueName: \"kubernetes.io/projected/afb0ba0b-8564-4f7d-889c-adcefd9da27a-kube-api-access-bx5mq\") pod \"cluster-samples-operator-7cdcfcf76c-spfsq\" (UID: \"afb0ba0b-8564-4f7d-889c-adcefd9da27a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq"
Sep 08 20:26:19.230800 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-bound-sa-token\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.230800 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230639    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-spfsq\" (UID: \"afb0ba0b-8564-4f7d-889c-adcefd9da27a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq"
Sep 08 20:26:19.230800 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230717    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d374121e-fdb4-4528-bee6-ffd573438cf1-config\") pod \"service-ca-operator-859f94c4b-qw8tl\" (UID: \"d374121e-fdb4-4528-bee6-ffd573438cf1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl"
Sep 08 20:26:19.230931 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.230875    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-nnl6n"]
Sep 08 20:26:19.233227 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.233209    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.233492 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.233470    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d374121e-fdb4-4528-bee6-ffd573438cf1-serving-cert\") pod \"service-ca-operator-859f94c4b-qw8tl\" (UID: \"d374121e-fdb4-4528-bee6-ffd573438cf1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl"
Sep 08 20:26:19.236580 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.236561    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 08 20:26:19.236679 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.236616    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 08 20:26:19.237244 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.237219    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 08 20:26:19.237336 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.237274    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 08 20:26:19.237336 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.237290    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-bd6p2\""
Sep 08 20:26:19.243562 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.243544    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-htbdg"]
Sep 08 20:26:19.245351 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.245332    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 08 20:26:19.245435 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.245422    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-htbdg"
Sep 08 20:26:19.247629 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.247613    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-h9b9x\""
Sep 08 20:26:19.248375 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.248355    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-nnl6n"]
Sep 08 20:26:19.250722 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.250706    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg7nh\" (UniqueName: \"kubernetes.io/projected/d374121e-fdb4-4528-bee6-ffd573438cf1-kube-api-access-cg7nh\") pod \"service-ca-operator-859f94c4b-qw8tl\" (UID: \"d374121e-fdb4-4528-bee6-ffd573438cf1\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl"
Sep 08 20:26:19.262719 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.262693    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-htbdg"]
Sep 08 20:26:19.331211 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331066    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6kmp\" (UniqueName: \"kubernetes.io/projected/5e7ff65d-5192-457e-aac7-8eef1a56d725-kube-api-access-b6kmp\") pod \"volume-data-source-validator-695c895bd6-glk4q\" (UID: \"5e7ff65d-5192-457e-aac7-8eef1a56d725\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-glk4q"
Sep 08 20:26:19.331389 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331208    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2129f9ae-e512-4e49-8f03-ec052fb52046-ca-trust-extracted\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.331389 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cl6hs\" (UID: \"d2ed3c01-5a4d-475b-84f3-97c7969c52da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:19.331389 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d59940d-9509-4afb-92bd-0a37c249aaa5-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.331389 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331302    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-spfsq\" (UID: \"afb0ba0b-8564-4f7d-889c-adcefd9da27a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq"
Sep 08 20:26:19.331389 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:19.331362    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 20:26:19.331389 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331373    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fm9m\" (UniqueName: \"kubernetes.io/projected/d2ed3c01-5a4d-475b-84f3-97c7969c52da-kube-api-access-5fm9m\") pod \"cluster-monitoring-operator-6cdcc96965-cl6hs\" (UID: \"d2ed3c01-5a4d-475b-84f3-97c7969c52da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:19.331693 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:19.331406    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 20:26:19.331693 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2129f9ae-e512-4e49-8f03-ec052fb52046-installation-pull-secrets\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.331693 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:19.331440    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls podName:d2ed3c01-5a4d-475b-84f3-97c7969c52da nodeName:}" failed. No retries permitted until 2026-09-08 20:26:19.831421032 +0000 UTC m=+53.378670510 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cl6hs" (UID: "d2ed3c01-5a4d-475b-84f3-97c7969c52da") : secret "cluster-monitoring-operator-tls" not found
Sep 08 20:26:19.331693 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:19.331468    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls podName:afb0ba0b-8564-4f7d-889c-adcefd9da27a nodeName:}" failed. No retries permitted until 2026-09-08 20:26:19.831451264 +0000 UTC m=+53.378700726 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-spfsq" (UID: "afb0ba0b-8564-4f7d-889c-adcefd9da27a") : secret "samples-operator-tls" not found
Sep 08 20:26:19.331693 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331511    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-certificates\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.331693 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2129f9ae-e512-4e49-8f03-ec052fb52046-image-registry-private-configuration\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.331693 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331568    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d59940d-9509-4afb-92bd-0a37c249aaa5-serving-cert\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.331693 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331624    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2129f9ae-e512-4e49-8f03-ec052fb52046-ca-trust-extracted\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.331693 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331614    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d59940d-9509-4afb-92bd-0a37c249aaa5-tmp\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.332134 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331761    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.332134 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331808    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx5mq\" (UniqueName: \"kubernetes.io/projected/afb0ba0b-8564-4f7d-889c-adcefd9da27a-kube-api-access-bx5mq\") pod \"cluster-samples-operator-7cdcfcf76c-spfsq\" (UID: \"afb0ba0b-8564-4f7d-889c-adcefd9da27a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq"
Sep 08 20:26:19.332134 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331839    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-bound-sa-token\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.332134 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331868    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d59940d-9509-4afb-92bd-0a37c249aaa5-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.332134 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:19.331900    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 20:26:19.332134 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:19.331915    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-596c57999-j8bwl: secret "image-registry-tls" not found
Sep 08 20:26:19.332134 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:19.331977    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls podName:2129f9ae-e512-4e49-8f03-ec052fb52046 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:19.831942959 +0000 UTC m=+53.379192442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls") pod "image-registry-596c57999-j8bwl" (UID: "2129f9ae-e512-4e49-8f03-ec052fb52046") : secret "image-registry-tls" not found
Sep 08 20:26:19.332134 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.331898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mtvk\" (UniqueName: \"kubernetes.io/projected/2d59940d-9509-4afb-92bd-0a37c249aaa5-kube-api-access-9mtvk\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.332134 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.332019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d2ed3c01-5a4d-475b-84f3-97c7969c52da-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-cl6hs\" (UID: \"d2ed3c01-5a4d-475b-84f3-97c7969c52da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:19.332134 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.332046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2129f9ae-e512-4e49-8f03-ec052fb52046-trusted-ca\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.332134 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.332058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-certificates\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.332134 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.332080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j72p\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-kube-api-access-2j72p\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.332628 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.332582    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2d59940d-9509-4afb-92bd-0a37c249aaa5-snapshots\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.332729 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.332678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94c49\" (UniqueName: \"kubernetes.io/projected/6fb58808-b701-4bab-b938-ab69074320e2-kube-api-access-94c49\") pod \"network-check-source-685456655f-htbdg\" (UID: \"6fb58808-b701-4bab-b938-ab69074320e2\") " pod="openshift-network-diagnostics/network-check-source-685456655f-htbdg"
Sep 08 20:26:19.333161 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.333137    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d2ed3c01-5a4d-475b-84f3-97c7969c52da-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-cl6hs\" (UID: \"d2ed3c01-5a4d-475b-84f3-97c7969c52da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:19.333617 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.333593    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2129f9ae-e512-4e49-8f03-ec052fb52046-trusted-ca\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.336631 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.334356    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2129f9ae-e512-4e49-8f03-ec052fb52046-image-registry-private-configuration\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.337432 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.337411    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl"
Sep 08 20:26:19.337566 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.337548    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2129f9ae-e512-4e49-8f03-ec052fb52046-installation-pull-secrets\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.345215 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.345186    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6kmp\" (UniqueName: \"kubernetes.io/projected/5e7ff65d-5192-457e-aac7-8eef1a56d725-kube-api-access-b6kmp\") pod \"volume-data-source-validator-695c895bd6-glk4q\" (UID: \"5e7ff65d-5192-457e-aac7-8eef1a56d725\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-glk4q"
Sep 08 20:26:19.352987 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.352952    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j72p\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-kube-api-access-2j72p\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.354309 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.354282    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-bound-sa-token\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.354856 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.354834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx5mq\" (UniqueName: \"kubernetes.io/projected/afb0ba0b-8564-4f7d-889c-adcefd9da27a-kube-api-access-bx5mq\") pod \"cluster-samples-operator-7cdcfcf76c-spfsq\" (UID: \"afb0ba0b-8564-4f7d-889c-adcefd9da27a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq"
Sep 08 20:26:19.355494 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.355476    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fm9m\" (UniqueName: \"kubernetes.io/projected/d2ed3c01-5a4d-475b-84f3-97c7969c52da-kube-api-access-5fm9m\") pod \"cluster-monitoring-operator-6cdcc96965-cl6hs\" (UID: \"d2ed3c01-5a4d-475b-84f3-97c7969c52da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:19.400577 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.400181    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:19.433551 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.433502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2d59940d-9509-4afb-92bd-0a37c249aaa5-snapshots\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.433551 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.433547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94c49\" (UniqueName: \"kubernetes.io/projected/6fb58808-b701-4bab-b938-ab69074320e2-kube-api-access-94c49\") pod \"network-check-source-685456655f-htbdg\" (UID: \"6fb58808-b701-4bab-b938-ab69074320e2\") " pod="openshift-network-diagnostics/network-check-source-685456655f-htbdg"
Sep 08 20:26:19.433795 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.433590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d59940d-9509-4afb-92bd-0a37c249aaa5-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.433795 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.433638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d59940d-9509-4afb-92bd-0a37c249aaa5-serving-cert\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.433795 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.433658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d59940d-9509-4afb-92bd-0a37c249aaa5-tmp\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.433795 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.433715    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d59940d-9509-4afb-92bd-0a37c249aaa5-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.433795 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.433741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mtvk\" (UniqueName: \"kubernetes.io/projected/2d59940d-9509-4afb-92bd-0a37c249aaa5-kube-api-access-9mtvk\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.434329 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.434307    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2d59940d-9509-4afb-92bd-0a37c249aaa5-snapshots\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.434329 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.434320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d59940d-9509-4afb-92bd-0a37c249aaa5-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.434445 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.434338    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2d59940d-9509-4afb-92bd-0a37c249aaa5-tmp\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.434712 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.434696    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d59940d-9509-4afb-92bd-0a37c249aaa5-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.436086 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.436066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d59940d-9509-4afb-92bd-0a37c249aaa5-serving-cert\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.444132 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.444108    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94c49\" (UniqueName: \"kubernetes.io/projected/6fb58808-b701-4bab-b938-ab69074320e2-kube-api-access-94c49\") pod \"network-check-source-685456655f-htbdg\" (UID: \"6fb58808-b701-4bab-b938-ab69074320e2\") " pod="openshift-network-diagnostics/network-check-source-685456655f-htbdg"
Sep 08 20:26:19.444238 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.444221    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mtvk\" (UniqueName: \"kubernetes.io/projected/2d59940d-9509-4afb-92bd-0a37c249aaa5-kube-api-access-9mtvk\") pod \"insights-operator-76bcd6ccf5-nnl6n\" (UID: \"2d59940d-9509-4afb-92bd-0a37c249aaa5\") " pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.444952 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.444939    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-glk4q"
Sep 08 20:26:19.493864 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.493824    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:19.545914 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.545885    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n"
Sep 08 20:26:19.557684 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.557670    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-htbdg"
Sep 08 20:26:19.605988 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.605911    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl"]
Sep 08 20:26:19.617768 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:26:19.617735    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd374121e_fdb4_4528_bee6_ffd573438cf1.slice/crio-4ae3e1973d9890448dced358f36d7c72446598f0183dbfc29c59fb58fde5e213 WatchSource:0}: Error finding container 4ae3e1973d9890448dced358f36d7c72446598f0183dbfc29c59fb58fde5e213: Status 404 returned error can't find the container with id 4ae3e1973d9890448dced358f36d7c72446598f0183dbfc29c59fb58fde5e213
Sep 08 20:26:19.711655 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.711622    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-glk4q"]
Sep 08 20:26:19.714472 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:26:19.714439    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e7ff65d_5192_457e_aac7_8eef1a56d725.slice/crio-3b6fbf1295bdd22e5ef0d01ef873649a5ec94b6a1c64b138475a151e1ca4b469 WatchSource:0}: Error finding container 3b6fbf1295bdd22e5ef0d01ef873649a5ec94b6a1c64b138475a151e1ca4b469: Status 404 returned error can't find the container with id 3b6fbf1295bdd22e5ef0d01ef873649a5ec94b6a1c64b138475a151e1ca4b469
Sep 08 20:26:19.817323 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.817236    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-nnl6n"]
Sep 08 20:26:19.819407 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:26:19.819379    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d59940d_9509_4afb_92bd_0a37c249aaa5.slice/crio-b176eee63da388c6c04f55297ffb9876f517df6bc0289e8186ba6fb9c0e4d983 WatchSource:0}: Error finding container b176eee63da388c6c04f55297ffb9876f517df6bc0289e8186ba6fb9c0e4d983: Status 404 returned error can't find the container with id b176eee63da388c6c04f55297ffb9876f517df6bc0289e8186ba6fb9c0e4d983
Sep 08 20:26:19.834529 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.834500    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-htbdg"]
Sep 08 20:26:19.837592 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.837561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cl6hs\" (UID: \"d2ed3c01-5a4d-475b-84f3-97c7969c52da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:19.837688 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.837596    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-spfsq\" (UID: \"afb0ba0b-8564-4f7d-889c-adcefd9da27a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq"
Sep 08 20:26:19.837688 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:26:19.837628    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb58808_b701_4bab_b938_ab69074320e2.slice/crio-700f3a7f8c786ad6e0266fafd537c9a577befc6fd4dc3f75b04a524d567e4ff0 WatchSource:0}: Error finding container 700f3a7f8c786ad6e0266fafd537c9a577befc6fd4dc3f75b04a524d567e4ff0: Status 404 returned error can't find the container with id 700f3a7f8c786ad6e0266fafd537c9a577befc6fd4dc3f75b04a524d567e4ff0
Sep 08 20:26:19.837688 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:19.837655    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:19.837802 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:19.837724    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 20:26:19.837802 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:19.837738    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 20:26:19.837802 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:19.837743    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 20:26:19.837802 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:19.837749    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-596c57999-j8bwl: secret "image-registry-tls" not found
Sep 08 20:26:19.837802 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:19.837790    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls podName:d2ed3c01-5a4d-475b-84f3-97c7969c52da nodeName:}" failed. No retries permitted until 2026-09-08 20:26:20.837771216 +0000 UTC m=+54.385020682 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cl6hs" (UID: "d2ed3c01-5a4d-475b-84f3-97c7969c52da") : secret "cluster-monitoring-operator-tls" not found
Sep 08 20:26:19.837985 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:19.837808    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls podName:afb0ba0b-8564-4f7d-889c-adcefd9da27a nodeName:}" failed. No retries permitted until 2026-09-08 20:26:20.837799292 +0000 UTC m=+54.385048757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-spfsq" (UID: "afb0ba0b-8564-4f7d-889c-adcefd9da27a") : secret "samples-operator-tls" not found
Sep 08 20:26:19.837985 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:19.837823    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls podName:2129f9ae-e512-4e49-8f03-ec052fb52046 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:20.837815004 +0000 UTC m=+54.385064464 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls") pod "image-registry-596c57999-j8bwl" (UID: "2129f9ae-e512-4e49-8f03-ec052fb52046") : secret "image-registry-tls" not found
Sep 08 20:26:20.152032 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:20.151995    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-htbdg" event={"ID":"6fb58808-b701-4bab-b938-ab69074320e2","Type":"ContainerStarted","Data":"700f3a7f8c786ad6e0266fafd537c9a577befc6fd4dc3f75b04a524d567e4ff0"}
Sep 08 20:26:20.153080 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:20.153058    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n" event={"ID":"2d59940d-9509-4afb-92bd-0a37c249aaa5","Type":"ContainerStarted","Data":"b176eee63da388c6c04f55297ffb9876f517df6bc0289e8186ba6fb9c0e4d983"}
Sep 08 20:26:20.154167 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:20.154146    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-glk4q" event={"ID":"5e7ff65d-5192-457e-aac7-8eef1a56d725","Type":"ContainerStarted","Data":"3b6fbf1295bdd22e5ef0d01ef873649a5ec94b6a1c64b138475a151e1ca4b469"}
Sep 08 20:26:20.155107 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:20.155088    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl" event={"ID":"d374121e-fdb4-4528-bee6-ffd573438cf1","Type":"ContainerStarted","Data":"4ae3e1973d9890448dced358f36d7c72446598f0183dbfc29c59fb58fde5e213"}
Sep 08 20:26:20.847737 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:20.847703    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:20.847737 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:20.847766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cl6hs\" (UID: \"d2ed3c01-5a4d-475b-84f3-97c7969c52da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:20.848163 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:20.847787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-spfsq\" (UID: \"afb0ba0b-8564-4f7d-889c-adcefd9da27a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq"
Sep 08 20:26:20.848163 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:20.847929    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 20:26:20.848163 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:20.847993    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 20:26:20.848163 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:20.848008    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-596c57999-j8bwl: secret "image-registry-tls" not found
Sep 08 20:26:20.848163 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:20.848009    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls podName:afb0ba0b-8564-4f7d-889c-adcefd9da27a nodeName:}" failed. No retries permitted until 2026-09-08 20:26:22.847990231 +0000 UTC m=+56.395239693 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-spfsq" (UID: "afb0ba0b-8564-4f7d-889c-adcefd9da27a") : secret "samples-operator-tls" not found
Sep 08 20:26:20.848163 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:20.848051    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls podName:2129f9ae-e512-4e49-8f03-ec052fb52046 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:22.848040245 +0000 UTC m=+56.395289710 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls") pod "image-registry-596c57999-j8bwl" (UID: "2129f9ae-e512-4e49-8f03-ec052fb52046") : secret "image-registry-tls" not found
Sep 08 20:26:20.848163 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:20.848105    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 20:26:20.848163 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:20.848135    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls podName:d2ed3c01-5a4d-475b-84f3-97c7969c52da nodeName:}" failed. No retries permitted until 2026-09-08 20:26:22.848125423 +0000 UTC m=+56.395374896 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cl6hs" (UID: "d2ed3c01-5a4d-475b-84f3-97c7969c52da") : secret "cluster-monitoring-operator-tls" not found
Sep 08 20:26:22.161805 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:22.161758    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-glk4q" event={"ID":"5e7ff65d-5192-457e-aac7-8eef1a56d725","Type":"ContainerStarted","Data":"fd422e1c0318319347b4897636371c898e74617bf7758a1287fae8f50d3ec37b"}
Sep 08 20:26:22.179025 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:22.178973    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-glk4q" podStartSLOduration=1.739100707 podStartE2EDuration="3.178941711s" totalImagesPullingTime="1.439841004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:22.177597708 +0000 UTC m=+55.724847188" watchObservedRunningTime="2026-09-08 20:26:22.178941711 +0000 UTC m=+55.726191203"
Sep 08 20:26:22.285112 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:22.285073    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:22.343666 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:22.343625    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:22.649993 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:22.649944    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:22.709250 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:22.709170    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:22.867255 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:22.867219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cl6hs\" (UID: \"d2ed3c01-5a4d-475b-84f3-97c7969c52da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:22.867255 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:22.867259    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-spfsq\" (UID: \"afb0ba0b-8564-4f7d-889c-adcefd9da27a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq"
Sep 08 20:26:22.867500 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:22.867372    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 20:26:22.867500 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:22.867377    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 20:26:22.867500 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:22.867458    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls podName:d2ed3c01-5a4d-475b-84f3-97c7969c52da nodeName:}" failed. No retries permitted until 2026-09-08 20:26:26.8674379 +0000 UTC m=+60.414687359 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cl6hs" (UID: "d2ed3c01-5a4d-475b-84f3-97c7969c52da") : secret "cluster-monitoring-operator-tls" not found
Sep 08 20:26:22.867500 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:22.867482    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls podName:afb0ba0b-8564-4f7d-889c-adcefd9da27a nodeName:}" failed. No retries permitted until 2026-09-08 20:26:26.867474611 +0000 UTC m=+60.414724069 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-spfsq" (UID: "afb0ba0b-8564-4f7d-889c-adcefd9da27a") : secret "samples-operator-tls" not found
Sep 08 20:26:22.867500 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:22.867497    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:22.867679 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:22.867605    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 20:26:22.867679 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:22.867614    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-596c57999-j8bwl: secret "image-registry-tls" not found
Sep 08 20:26:22.867679 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:22.867644    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls podName:2129f9ae-e512-4e49-8f03-ec052fb52046 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:26.867637157 +0000 UTC m=+60.414886615 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls") pod "image-registry-596c57999-j8bwl" (UID: "2129f9ae-e512-4e49-8f03-ec052fb52046") : secret "image-registry-tls" not found
Sep 08 20:26:24.169750 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:24.169714    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n" event={"ID":"2d59940d-9509-4afb-92bd-0a37c249aaa5","Type":"ContainerStarted","Data":"10e8ef5c75831f57eb2458729befded000db3b42db58c3f4565f26a8db237968"}
Sep 08 20:26:24.171691 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:24.171659    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl" event={"ID":"d374121e-fdb4-4528-bee6-ffd573438cf1","Type":"ContainerStarted","Data":"82523636fcc08bfd4376032229170d9110781a61367c1ff213825dcaca801d3e"}
Sep 08 20:26:24.173602 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:24.173575    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-htbdg" event={"ID":"6fb58808-b701-4bab-b938-ab69074320e2","Type":"ContainerStarted","Data":"fc0232ba2e71a8fb8db2237f674c730582c663b3306f5409c491b0d4cea790b8"}
Sep 08 20:26:24.188934 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:24.187697    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n" podStartSLOduration=1.606874039 podStartE2EDuration="5.187678485s" totalImagesPullingTime="3.580804446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:24.187079043 +0000 UTC m=+57.734328525" watchObservedRunningTime="2026-09-08 20:26:24.187678485 +0000 UTC m=+57.734927968"
Sep 08 20:26:24.203436 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:24.203384    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl" podStartSLOduration=1.417015046 podStartE2EDuration="5.203366829s" totalImagesPullingTime="3.786351783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:24.202154861 +0000 UTC m=+57.749404343" watchObservedRunningTime="2026-09-08 20:26:24.203366829 +0000 UTC m=+57.750616325"
Sep 08 20:26:24.218089 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:24.218012    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-htbdg" podStartSLOduration=1.644984057 podStartE2EDuration="5.217994196s" totalImagesPullingTime="3.573010139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:24.216823911 +0000 UTC m=+57.764073393" watchObservedRunningTime="2026-09-08 20:26:24.217994196 +0000 UTC m=+57.765243679"
Sep 08 20:26:24.330347 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:24.330295    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:24.430801 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:24.430704    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:24.785559 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:24.785466    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:24.851901 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:24.851867    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:25.029193 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.029155    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:25.158709 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.158676    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:25.487062 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.486987    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hnccs"]
Sep 08 20:26:25.497860 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.497826    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:25.515857 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.515826    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hnccs"]
Sep 08 20:26:25.516010 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.515942    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:25.518359 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.518334    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 08 20:26:25.518488 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.518465    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 08 20:26:25.518488 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.518478    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4gd8m\""
Sep 08 20:26:25.567693 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.567661    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:25.589925 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.589893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d4d58a5e-85e0-4276-a8d5-869820c7f439-crio-socket\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:25.590092 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.589971    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4d58a5e-85e0-4276-a8d5-869820c7f439-data-volume\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:25.590092 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.590022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:25.590092 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.590046    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmk66\" (UniqueName: \"kubernetes.io/projected/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-api-access-vmk66\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:25.590205 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.590099    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:25.690809 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.690769    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:25.691028 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.690857    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d4d58a5e-85e0-4276-a8d5-869820c7f439-crio-socket\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:25.691028 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:25.690904    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 20:26:25.691028 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.690931    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4d58a5e-85e0-4276-a8d5-869820c7f439-data-volume\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:25.691028 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.691004    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d4d58a5e-85e0-4276-a8d5-869820c7f439-crio-socket\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:25.691028 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:25.691008    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm podName:d4d58a5e-85e0-4276-a8d5-869820c7f439 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:26.190986544 +0000 UTC m=+59.738236004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hnccs" (UID: "d4d58a5e-85e0-4276-a8d5-869820c7f439") : configmap "kube-rbac-proxy" not found
Sep 08 20:26:25.691301 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.691072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:25.691301 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.691091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmk66\" (UniqueName: \"kubernetes.io/projected/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-api-access-vmk66\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:25.691301 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:25.691212    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:25.691301 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:25.691273    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls podName:d4d58a5e-85e0-4276-a8d5-869820c7f439 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:26.19125744 +0000 UTC m=+59.738506902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hnccs" (UID: "d4d58a5e-85e0-4276-a8d5-869820c7f439") : secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:25.691441 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.691314    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4d58a5e-85e0-4276-a8d5-869820c7f439-data-volume\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:25.701828 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:25.701800    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmk66\" (UniqueName: \"kubernetes.io/projected/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-api-access-vmk66\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:26.194715 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:26.194674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:26.194715 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:26.194722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:26.195001 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:26.194799    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 20:26:26.195001 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:26.194840    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:26.195001 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:26.194855    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm podName:d4d58a5e-85e0-4276-a8d5-869820c7f439 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:27.194841077 +0000 UTC m=+60.742090536 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hnccs" (UID: "d4d58a5e-85e0-4276-a8d5-869820c7f439") : configmap "kube-rbac-proxy" not found
Sep 08 20:26:26.195001 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:26.194913    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls podName:d4d58a5e-85e0-4276-a8d5-869820c7f439 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:27.194896448 +0000 UTC m=+60.742145922 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hnccs" (UID: "d4d58a5e-85e0-4276-a8d5-869820c7f439") : secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:26.673530 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:26.673495    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:26.753706 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:26.753669    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:26.832616 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:26.832583    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:26.899807 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:26.899777    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-spfsq\" (UID: \"afb0ba0b-8564-4f7d-889c-adcefd9da27a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq"
Sep 08 20:26:26.899988 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:26.899851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:26.899988 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:26.899911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cl6hs\" (UID: \"d2ed3c01-5a4d-475b-84f3-97c7969c52da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:26.899988 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:26.899912    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 20:26:26.900147 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:26.899998    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 20:26:26.900147 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:26.900002    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls podName:afb0ba0b-8564-4f7d-889c-adcefd9da27a nodeName:}" failed. No retries permitted until 2026-09-08 20:26:34.89998174 +0000 UTC m=+68.447231201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-spfsq" (UID: "afb0ba0b-8564-4f7d-889c-adcefd9da27a") : secret "samples-operator-tls" not found
Sep 08 20:26:26.900147 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:26.900010    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 20:26:26.900147 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:26.900027    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-596c57999-j8bwl: secret "image-registry-tls" not found
Sep 08 20:26:26.900147 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:26.900045    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls podName:d2ed3c01-5a4d-475b-84f3-97c7969c52da nodeName:}" failed. No retries permitted until 2026-09-08 20:26:34.900034165 +0000 UTC m=+68.447283624 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cl6hs" (UID: "d2ed3c01-5a4d-475b-84f3-97c7969c52da") : secret "cluster-monitoring-operator-tls" not found
Sep 08 20:26:26.900147 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:26.900081    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls podName:2129f9ae-e512-4e49-8f03-ec052fb52046 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:34.900066984 +0000 UTC m=+68.447316458 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls") pod "image-registry-596c57999-j8bwl" (UID: "2129f9ae-e512-4e49-8f03-ec052fb52046") : secret "image-registry-tls" not found
Sep 08 20:26:27.090422 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:27.090348    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x29fz"
Sep 08 20:26:27.202336 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:27.202289    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:27.202487 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:27.202370    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:27.202487 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:27.202428    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:27.202610 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:27.202491    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls podName:d4d58a5e-85e0-4276-a8d5-869820c7f439 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:29.202472636 +0000 UTC m=+62.749722112 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hnccs" (UID: "d4d58a5e-85e0-4276-a8d5-869820c7f439") : secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:27.202610 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:27.202589    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 20:26:27.202710 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:27.202655    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm podName:d4d58a5e-85e0-4276-a8d5-869820c7f439 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:29.202634677 +0000 UTC m=+62.749884140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hnccs" (UID: "d4d58a5e-85e0-4276-a8d5-869820c7f439") : configmap "kube-rbac-proxy" not found
Sep 08 20:26:27.261776 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:27.261750    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cljhb_de02ff1a-312a-4069-b121-dff1aa3b9354/dns-node-resolver/0.log"
Sep 08 20:26:27.285663 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:27.285636    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:27.387339 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:27.387305    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:27.486589 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:27.486554    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:27.657557 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:27.657477    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:28.233516 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:28.233490    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gc9sv_b691d62f-ea1a-43a9-909e-ab03a8bf9435/node-ca/0.log"
Sep 08 20:26:29.034782 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:29.034754    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ck7k5_0b838c12-7a7d-432e-ad79-d05e1104643f/serve-healthcheck-canary/0.log"
Sep 08 20:26:29.218941 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:29.218898    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:29.219142 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:29.219071    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 20:26:29.219142 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:29.219137    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm podName:d4d58a5e-85e0-4276-a8d5-869820c7f439 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:33.219121809 +0000 UTC m=+66.766371268 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hnccs" (UID: "d4d58a5e-85e0-4276-a8d5-869820c7f439") : configmap "kube-rbac-proxy" not found
Sep 08 20:26:29.219241 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:29.219179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:29.219294 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:29.219282    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:29.219335 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:29.219321    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls podName:d4d58a5e-85e0-4276-a8d5-869820c7f439 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:33.219309441 +0000 UTC m=+66.766558902 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hnccs" (UID: "d4d58a5e-85e0-4276-a8d5-869820c7f439") : secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:30.892358 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:30.892314    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:31.514448 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:31.514409    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:31.649833 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:31.649797    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:31.712459 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:31.712429    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:31.815354 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:31.815241    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:31.922139 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:31.922088    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:32.419663 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:32.419620    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:32.647391 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:32.647357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret\") pod \"global-pull-secret-syncer-fkjc4\" (UID: \"a9304897-72a4-4daa-aa6e-9397a440502e\") " pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:26:32.647573 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:32.647414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdnvw\" (UniqueName: \"kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw\") pod \"network-check-target-vjhfp\" (UID: \"91b9ef98-2cf6-4b27-97b5-01662e8c4058\") " pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:26:32.649649 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:32.649626    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 20:26:32.649925 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:32.649906    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdnvw\" (UniqueName: \"kubernetes.io/projected/91b9ef98-2cf6-4b27-97b5-01662e8c4058-kube-api-access-jdnvw\") pod \"network-check-target-vjhfp\" (UID: \"91b9ef98-2cf6-4b27-97b5-01662e8c4058\") " pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:26:32.660517 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:32.660486    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9304897-72a4-4daa-aa6e-9397a440502e-original-pull-secret\") pod \"global-pull-secret-syncer-fkjc4\" (UID: \"a9304897-72a4-4daa-aa6e-9397a440502e\") " pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:26:32.748430 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:32.748336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:26:32.748597 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:32.748518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs\") pod \"network-metrics-daemon-vlx4d\" (UID: \"642122c1-c2a9-4fe3-aade-12ac05cecf17\") " pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:26:32.748597 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:32.748518    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 20:26:32.748702 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:32.748617    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls podName:34b69620-f3c4-471b-85d5-887e22e90910 nodeName:}" failed. No retries permitted until 2026-09-08 20:27:04.748600876 +0000 UTC m=+98.295850334 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls") pod "dns-default-75k8c" (UID: "34b69620-f3c4-471b-85d5-887e22e90910") : secret "dns-default-metrics-tls" not found
Sep 08 20:26:32.751039 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:32.751017    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 20:26:32.758737 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:32.758715    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 08 20:26:32.758839 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:32.758780    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs podName:642122c1-c2a9-4fe3-aade-12ac05cecf17 nodeName:}" failed. No retries permitted until 2026-09-08 20:27:36.758760642 +0000 UTC m=+130.306010106 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs") pod "network-metrics-daemon-vlx4d" (UID: "642122c1-c2a9-4fe3-aade-12ac05cecf17") : secret "metrics-daemon-secret" not found
Sep 08 20:26:32.823727 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:32.823695    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:32.828617 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:32.828595    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fkjc4"
Sep 08 20:26:32.835547 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:32.835523    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2nzg7\""
Sep 08 20:26:32.844318 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:32.844290    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:26:33.119921 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:33.119819    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fkjc4"]
Sep 08 20:26:33.122505 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:26:33.122478    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9304897_72a4_4daa_aa6e_9397a440502e.slice/crio-8587a4ce3ec47b150b85c589f512fd34b7968ea737410f7a9397c57df939d7ad WatchSource:0}: Error finding container 8587a4ce3ec47b150b85c589f512fd34b7968ea737410f7a9397c57df939d7ad: Status 404 returned error can't find the container with id 8587a4ce3ec47b150b85c589f512fd34b7968ea737410f7a9397c57df939d7ad
Sep 08 20:26:33.143701 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:33.143671    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vjhfp"]
Sep 08 20:26:33.146500 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:26:33.146467    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91b9ef98_2cf6_4b27_97b5_01662e8c4058.slice/crio-0327edd60f952d755e6e33b14086d2e0edc91c3571ea26ca9b4ecd29616a5448 WatchSource:0}: Error finding container 0327edd60f952d755e6e33b14086d2e0edc91c3571ea26ca9b4ecd29616a5448: Status 404 returned error can't find the container with id 0327edd60f952d755e6e33b14086d2e0edc91c3571ea26ca9b4ecd29616a5448
Sep 08 20:26:33.198616 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:33.198586    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vjhfp" event={"ID":"91b9ef98-2cf6-4b27-97b5-01662e8c4058","Type":"ContainerStarted","Data":"0327edd60f952d755e6e33b14086d2e0edc91c3571ea26ca9b4ecd29616a5448"}
Sep 08 20:26:33.199662 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:33.199638    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fkjc4" event={"ID":"a9304897-72a4-4daa-aa6e-9397a440502e","Type":"ContainerStarted","Data":"8587a4ce3ec47b150b85c589f512fd34b7968ea737410f7a9397c57df939d7ad"}
Sep 08 20:26:33.253663 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:33.253634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:33.253807 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:33.253749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:33.253807 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:33.253768    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 20:26:33.253908 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:33.253848    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:33.253908 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:33.253849    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm podName:d4d58a5e-85e0-4276-a8d5-869820c7f439 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:41.253826942 +0000 UTC m=+74.801076428 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hnccs" (UID: "d4d58a5e-85e0-4276-a8d5-869820c7f439") : configmap "kube-rbac-proxy" not found
Sep 08 20:26:33.253908 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:33.253896    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls podName:d4d58a5e-85e0-4276-a8d5-869820c7f439 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:41.253882806 +0000 UTC m=+74.801132265 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hnccs" (UID: "d4d58a5e-85e0-4276-a8d5-869820c7f439") : secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:33.412071 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:33.412038    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:34.013538 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:34.013495    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:34.208453 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:34.208414    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vjhfp" event={"ID":"91b9ef98-2cf6-4b27-97b5-01662e8c4058","Type":"ContainerStarted","Data":"89a5a73eae8fd837ad059cf9c589aef0a91b57922ad8ab0918c64319fb8cf2af"}
Sep 08 20:26:34.208863 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:34.208556    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:26:34.210040 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:34.210011    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:34.224036 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:34.223983    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vjhfp" podStartSLOduration=67.223949172 podStartE2EDuration="1m7.223949172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:25:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:34.223752342 +0000 UTC m=+67.771001823" watchObservedRunningTime="2026-09-08 20:26:34.223949172 +0000 UTC m=+67.771198654"
Sep 08 20:26:34.610093 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:34.610046    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:34.820911 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:34.820877    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:34.969342 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:34.969307    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls\") pod \"image-registry-596c57999-j8bwl\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") " pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:34.969518 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:34.969389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cl6hs\" (UID: \"d2ed3c01-5a4d-475b-84f3-97c7969c52da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:34.969518 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:34.969420    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-spfsq\" (UID: \"afb0ba0b-8564-4f7d-889c-adcefd9da27a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq"
Sep 08 20:26:34.969518 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:34.969461    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 20:26:34.969518 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:34.969481    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-596c57999-j8bwl: secret "image-registry-tls" not found
Sep 08 20:26:34.969702 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:34.969540    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls podName:2129f9ae-e512-4e49-8f03-ec052fb52046 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:50.969523181 +0000 UTC m=+84.516772660 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls") pod "image-registry-596c57999-j8bwl" (UID: "2129f9ae-e512-4e49-8f03-ec052fb52046") : secret "image-registry-tls" not found
Sep 08 20:26:34.969702 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:34.969541    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 20:26:34.969702 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:34.969607    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls podName:d2ed3c01-5a4d-475b-84f3-97c7969c52da nodeName:}" failed. No retries permitted until 2026-09-08 20:26:50.969588029 +0000 UTC m=+84.516837490 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cl6hs" (UID: "d2ed3c01-5a4d-475b-84f3-97c7969c52da") : secret "cluster-monitoring-operator-tls" not found
Sep 08 20:26:34.972077 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:34.972047    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afb0ba0b-8564-4f7d-889c-adcefd9da27a-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-spfsq\" (UID: \"afb0ba0b-8564-4f7d-889c-adcefd9da27a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq"
Sep 08 20:26:35.057467 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:35.057431    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-bpqh8\""
Sep 08 20:26:35.065611 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:35.065583    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq"
Sep 08 20:26:35.214323 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:35.214290    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:35.270325 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:35.270237    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:35.341686 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:35.341652    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq"]
Sep 08 20:26:35.414984 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:35.414938    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:35.615541 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:35.615417    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:35.675446 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:35.675405    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:36.214488 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:36.214453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:36.218017 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:36.217986    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq" event={"ID":"afb0ba0b-8564-4f7d-889c-adcefd9da27a","Type":"ContainerStarted","Data":"460de6b3f7407a7d2c5cba96f8ba89f52a98fb8468715576b861f5963d959d53"}
Sep 08 20:26:36.809591 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:36.809550    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:37.011816 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:37.011774    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:37.230546 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:37.228005    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fkjc4" event={"ID":"a9304897-72a4-4daa-aa6e-9397a440502e","Type":"ContainerStarted","Data":"7ef7a4832520400c7a2e688f53676989399e1e7bd93b300d4746f7b682757e89"}
Sep 08 20:26:37.244353 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:37.244291    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fkjc4" podStartSLOduration=66.419307947 podStartE2EDuration="1m10.244270925s" totalImagesPullingTime="3.824962978s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:37.242938464 +0000 UTC m=+70.790187944" watchObservedRunningTime="2026-09-08 20:26:37.244270925 +0000 UTC m=+70.791520406"
Sep 08 20:26:37.418180 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:37.418144    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:37.622105 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:37.622023    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:37.723503 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:37.723455    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:37.925915 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:37.925879    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:38.027850 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:38.027807    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:38.232983 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:38.232931    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq" event={"ID":"afb0ba0b-8564-4f7d-889c-adcefd9da27a","Type":"ContainerStarted","Data":"17883f67ac526d916f5775ef3c89091d1fdeb1a5e123a04675848071dda617cb"}
Sep 08 20:26:38.233419 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:38.233005    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq" event={"ID":"afb0ba0b-8564-4f7d-889c-adcefd9da27a","Type":"ContainerStarted","Data":"2d98c94d32411841c7201accdd5303d45b8faf1fd0ab6c6050ebb213ae597b57"}
Sep 08 20:26:38.248621 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:38.248548    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-spfsq" podStartSLOduration=16.559540815 podStartE2EDuration="19.248533573s" totalImagesPullingTime="2.688992758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:38.247759019 +0000 UTC m=+71.795008499" watchObservedRunningTime="2026-09-08 20:26:38.248533573 +0000 UTC m=+71.795783116"
Sep 08 20:26:38.637779 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:38.637747    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:39.013061 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:39.012977    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:39.410125 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:39.410081    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:39.559629 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:39.559599    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:39.621772 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:39.621743    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:40.014359 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:40.014313    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:40.414816 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:40.414784    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:41.021018 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:41.020981    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:41.323184 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:41.323096    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:41.323332 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:41.323209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:41.323332 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:41.323305    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:41.323403 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:41.323361    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls podName:d4d58a5e-85e0-4276-a8d5-869820c7f439 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:57.32334552 +0000 UTC m=+90.870594979 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hnccs" (UID: "d4d58a5e-85e0-4276-a8d5-869820c7f439") : secret "insights-runtime-extractor-tls" not found
Sep 08 20:26:41.323628 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:41.323609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4d58a5e-85e0-4276-a8d5-869820c7f439-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:41.424245 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:41.424194    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:41.808866 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:41.808831    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:41.932179 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:41.932144    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:42.035639 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:42.035595    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:42.088219 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:42.088135    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:42.220034 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:42.219981    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:42.690764 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:42.690724    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:42.791269 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:42.791240    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:42.892283 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:42.892249    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:42.996875 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:42.996781    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:43.366097 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:43.366012    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:43.767709 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:43.767673    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:43.820412 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:43.820372    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:44.369117 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:44.369081    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:44.568808 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:44.568773    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:44.775836 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:44.775807    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:44.881727 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:44.881693    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:45.192131 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:45.192096    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:45.245530 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:45.245491    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:47.398092 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.398056    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d"]
Sep 08 20:26:47.400614 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.400598    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d"
Sep 08 20:26:47.404623 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.404600    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 08 20:26:47.404722 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.404680    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 08 20:26:47.406334 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.406320    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 08 20:26:47.409336 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.409319    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 08 20:26:47.425229 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.425200    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d"]
Sep 08 20:26:47.470159 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.470110    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf7xz\" (UniqueName: \"kubernetes.io/projected/d07d64de-c856-4b22-82c7-fe1f891d6c8a-kube-api-access-xf7xz\") pod \"klusterlet-addon-workmgr-6c75987765-z862d\" (UID: \"d07d64de-c856-4b22-82c7-fe1f891d6c8a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d"
Sep 08 20:26:47.470159 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.470157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d07d64de-c856-4b22-82c7-fe1f891d6c8a-tmp\") pod \"klusterlet-addon-workmgr-6c75987765-z862d\" (UID: \"d07d64de-c856-4b22-82c7-fe1f891d6c8a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d"
Sep 08 20:26:47.470402 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.470245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d07d64de-c856-4b22-82c7-fe1f891d6c8a-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c75987765-z862d\" (UID: \"d07d64de-c856-4b22-82c7-fe1f891d6c8a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d"
Sep 08 20:26:47.486385 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.486348    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-596c57999-j8bwl"]
Sep 08 20:26:47.486631 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:47.486609    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-596c57999-j8bwl" podUID="2129f9ae-e512-4e49-8f03-ec052fb52046"
Sep 08 20:26:47.521086 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.521044    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:47.525770 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.525732    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-79f5ff44f6-sgnmg"]
Sep 08 20:26:47.527712 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.527698    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.545668 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.545641    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79f5ff44f6-sgnmg"]
Sep 08 20:26:47.570943 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.570906    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf7xz\" (UniqueName: \"kubernetes.io/projected/d07d64de-c856-4b22-82c7-fe1f891d6c8a-kube-api-access-xf7xz\") pod \"klusterlet-addon-workmgr-6c75987765-z862d\" (UID: \"d07d64de-c856-4b22-82c7-fe1f891d6c8a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d"
Sep 08 20:26:47.571153 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.570947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d07d64de-c856-4b22-82c7-fe1f891d6c8a-tmp\") pod \"klusterlet-addon-workmgr-6c75987765-z862d\" (UID: \"d07d64de-c856-4b22-82c7-fe1f891d6c8a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d"
Sep 08 20:26:47.571153 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.571042    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d07d64de-c856-4b22-82c7-fe1f891d6c8a-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c75987765-z862d\" (UID: \"d07d64de-c856-4b22-82c7-fe1f891d6c8a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d"
Sep 08 20:26:47.571409 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.571383    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d07d64de-c856-4b22-82c7-fe1f891d6c8a-tmp\") pod \"klusterlet-addon-workmgr-6c75987765-z862d\" (UID: \"d07d64de-c856-4b22-82c7-fe1f891d6c8a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d"
Sep 08 20:26:47.573708 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.573683    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d07d64de-c856-4b22-82c7-fe1f891d6c8a-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c75987765-z862d\" (UID: \"d07d64de-c856-4b22-82c7-fe1f891d6c8a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d"
Sep 08 20:26:47.594451 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.594414    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf7xz\" (UniqueName: \"kubernetes.io/projected/d07d64de-c856-4b22-82c7-fe1f891d6c8a-kube-api-access-xf7xz\") pod \"klusterlet-addon-workmgr-6c75987765-z862d\" (UID: \"d07d64de-c856-4b22-82c7-fe1f891d6c8a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d"
Sep 08 20:26:47.615565 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.615517    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:47.672007 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.671894    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52258924-6ad4-4c1b-bb14-1848b11d6133-trusted-ca\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.672007 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.671934    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbc8r\" (UniqueName: \"kubernetes.io/projected/52258924-6ad4-4c1b-bb14-1848b11d6133-kube-api-access-sbc8r\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.672007 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.672000    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52258924-6ad4-4c1b-bb14-1848b11d6133-registry-tls\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.672234 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.672021    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52258924-6ad4-4c1b-bb14-1848b11d6133-installation-pull-secrets\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.672234 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.672093    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52258924-6ad4-4c1b-bb14-1848b11d6133-image-registry-private-configuration\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.672234 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.672181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52258924-6ad4-4c1b-bb14-1848b11d6133-ca-trust-extracted\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.672234 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.672204    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52258924-6ad4-4c1b-bb14-1848b11d6133-registry-certificates\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.672234 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.672222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52258924-6ad4-4c1b-bb14-1848b11d6133-bound-sa-token\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.711014 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.710979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d"
Sep 08 20:26:47.726406 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.726362    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:47.773618 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.773579    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52258924-6ad4-4c1b-bb14-1848b11d6133-ca-trust-extracted\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.773725 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.773638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52258924-6ad4-4c1b-bb14-1848b11d6133-registry-certificates\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.773725 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.773669    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52258924-6ad4-4c1b-bb14-1848b11d6133-bound-sa-token\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.773814 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.773727    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52258924-6ad4-4c1b-bb14-1848b11d6133-trusted-ca\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.773814 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.773755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbc8r\" (UniqueName: \"kubernetes.io/projected/52258924-6ad4-4c1b-bb14-1848b11d6133-kube-api-access-sbc8r\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.773814 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.773801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52258924-6ad4-4c1b-bb14-1848b11d6133-registry-tls\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.773946 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.773829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52258924-6ad4-4c1b-bb14-1848b11d6133-installation-pull-secrets\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.773946 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.773878    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52258924-6ad4-4c1b-bb14-1848b11d6133-image-registry-private-configuration\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.774128 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.774106    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52258924-6ad4-4c1b-bb14-1848b11d6133-ca-trust-extracted\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.774792 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.774763    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52258924-6ad4-4c1b-bb14-1848b11d6133-registry-certificates\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.775220 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.775169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52258924-6ad4-4c1b-bb14-1848b11d6133-trusted-ca\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.777159 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.776710    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52258924-6ad4-4c1b-bb14-1848b11d6133-registry-tls\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.777159 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.777118    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52258924-6ad4-4c1b-bb14-1848b11d6133-image-registry-private-configuration\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.778468 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.778429    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52258924-6ad4-4c1b-bb14-1848b11d6133-installation-pull-secrets\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.783400 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.783380    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbc8r\" (UniqueName: \"kubernetes.io/projected/52258924-6ad4-4c1b-bb14-1848b11d6133-kube-api-access-sbc8r\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.784651 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.784627    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52258924-6ad4-4c1b-bb14-1848b11d6133-bound-sa-token\") pod \"image-registry-79f5ff44f6-sgnmg\" (UID: \"52258924-6ad4-4c1b-bb14-1848b11d6133\") " pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.819197 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.819167    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:47.838777 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.838756    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nwnm8\""
Sep 08 20:26:47.847842 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.847818    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:47.978797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:47.978711    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d"]
Sep 08 20:26:47.982430 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:26:47.982395    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd07d64de_c856_4b22_82c7_fe1f891d6c8a.slice/crio-ac4533f38b9abbf6bee49de9db53938821d7f132c88e672a5ba39ee5eb517dbd WatchSource:0}: Error finding container ac4533f38b9abbf6bee49de9db53938821d7f132c88e672a5ba39ee5eb517dbd: Status 404 returned error can't find the container with id ac4533f38b9abbf6bee49de9db53938821d7f132c88e672a5ba39ee5eb517dbd
Sep 08 20:26:48.136360 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.136330    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79f5ff44f6-sgnmg"]
Sep 08 20:26:48.139382 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:26:48.139358    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52258924_6ad4_4c1b_bb14_1848b11d6133.slice/crio-7bd446b3e4a75e2f0ba6dbdff0cfdcf957d6640e91b55c3bc5781867616c80b4 WatchSource:0}: Error finding container 7bd446b3e4a75e2f0ba6dbdff0cfdcf957d6640e91b55c3bc5781867616c80b4: Status 404 returned error can't find the container with id 7bd446b3e4a75e2f0ba6dbdff0cfdcf957d6640e91b55c3bc5781867616c80b4
Sep 08 20:26:48.267021 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.266921    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d" event={"ID":"d07d64de-c856-4b22-82c7-fe1f891d6c8a","Type":"ContainerStarted","Data":"ac4533f38b9abbf6bee49de9db53938821d7f132c88e672a5ba39ee5eb517dbd"}
Sep 08 20:26:48.268609 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.268582    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg" event={"ID":"52258924-6ad4-4c1b-bb14-1848b11d6133","Type":"ContainerStarted","Data":"69324007bce337d3784f4d1b0c6c72e20ccaa00c6d9ff0a4f7ff84a4bba5695a"}
Sep 08 20:26:48.268721 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.268617    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:48.268721 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.268635    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg" event={"ID":"52258924-6ad4-4c1b-bb14-1848b11d6133","Type":"ContainerStarted","Data":"7bd446b3e4a75e2f0ba6dbdff0cfdcf957d6640e91b55c3bc5781867616c80b4"}
Sep 08 20:26:48.274097 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.274074    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:48.304002 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.303930    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg" podStartSLOduration=1.303914043 podStartE2EDuration="1.303914043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:48.303494889 +0000 UTC m=+81.850744371" watchObservedRunningTime="2026-09-08 20:26:48.303914043 +0000 UTC m=+81.851163525"
Sep 08 20:26:48.379684 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.379644    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2129f9ae-e512-4e49-8f03-ec052fb52046-installation-pull-secrets\") pod \"2129f9ae-e512-4e49-8f03-ec052fb52046\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") "
Sep 08 20:26:48.379684 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.379682    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-bound-sa-token\") pod \"2129f9ae-e512-4e49-8f03-ec052fb52046\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") "
Sep 08 20:26:48.379931 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.379722    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2j72p\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-kube-api-access-2j72p\") pod \"2129f9ae-e512-4e49-8f03-ec052fb52046\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") "
Sep 08 20:26:48.379931 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.379746    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2129f9ae-e512-4e49-8f03-ec052fb52046-trusted-ca\") pod \"2129f9ae-e512-4e49-8f03-ec052fb52046\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") "
Sep 08 20:26:48.379931 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.379765    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2129f9ae-e512-4e49-8f03-ec052fb52046-ca-trust-extracted\") pod \"2129f9ae-e512-4e49-8f03-ec052fb52046\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") "
Sep 08 20:26:48.379931 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.379868    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2129f9ae-e512-4e49-8f03-ec052fb52046-image-registry-private-configuration\") pod \"2129f9ae-e512-4e49-8f03-ec052fb52046\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") "
Sep 08 20:26:48.379931 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.379924    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-certificates\") pod \"2129f9ae-e512-4e49-8f03-ec052fb52046\" (UID: \"2129f9ae-e512-4e49-8f03-ec052fb52046\") "
Sep 08 20:26:48.380237 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.380087    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2129f9ae-e512-4e49-8f03-ec052fb52046-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2129f9ae-e512-4e49-8f03-ec052fb52046" (UID: "2129f9ae-e512-4e49-8f03-ec052fb52046"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:26:48.380237 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.380190    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2129f9ae-e512-4e49-8f03-ec052fb52046-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2129f9ae-e512-4e49-8f03-ec052fb52046" (UID: "2129f9ae-e512-4e49-8f03-ec052fb52046"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:26:48.380412 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.380383    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2129f9ae-e512-4e49-8f03-ec052fb52046" (UID: "2129f9ae-e512-4e49-8f03-ec052fb52046"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:26:48.381066 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.380801    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2129f9ae-e512-4e49-8f03-ec052fb52046-trusted-ca\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:26:48.381066 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.380827    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2129f9ae-e512-4e49-8f03-ec052fb52046-ca-trust-extracted\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:26:48.381066 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.380843    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-certificates\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:26:48.382378 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.382356    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2129f9ae-e512-4e49-8f03-ec052fb52046-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2129f9ae-e512-4e49-8f03-ec052fb52046" (UID: "2129f9ae-e512-4e49-8f03-ec052fb52046"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:26:48.382475 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.382437    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2129f9ae-e512-4e49-8f03-ec052fb52046-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2129f9ae-e512-4e49-8f03-ec052fb52046" (UID: "2129f9ae-e512-4e49-8f03-ec052fb52046"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:26:48.382475 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.382467    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2129f9ae-e512-4e49-8f03-ec052fb52046" (UID: "2129f9ae-e512-4e49-8f03-ec052fb52046"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:26:48.382611 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.382596    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-kube-api-access-2j72p" (OuterVolumeSpecName: "kube-api-access-2j72p") pod "2129f9ae-e512-4e49-8f03-ec052fb52046" (UID: "2129f9ae-e512-4e49-8f03-ec052fb52046"). InnerVolumeSpecName "kube-api-access-2j72p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:26:48.481221 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.481184    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2129f9ae-e512-4e49-8f03-ec052fb52046-image-registry-private-configuration\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:26:48.481221 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.481210    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2129f9ae-e512-4e49-8f03-ec052fb52046-installation-pull-secrets\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:26:48.481221 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.481220    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-bound-sa-token\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:26:48.481221 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:48.481229    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2j72p\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-kube-api-access-2j72p\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:26:49.272979 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:49.272793    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-596c57999-j8bwl"
Sep 08 20:26:49.272979 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:49.272921    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:26:49.300951 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:49.300914    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-596c57999-j8bwl"]
Sep 08 20:26:49.303255 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:49.303229    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-596c57999-j8bwl"]
Sep 08 20:26:49.389214 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:49.389176    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2129f9ae-e512-4e49-8f03-ec052fb52046-registry-tls\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:26:50.715724 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:50.715690    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:50.924718 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:50.924683    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2129f9ae-e512-4e49-8f03-ec052fb52046" path="/var/lib/kubelet/pods/2129f9ae-e512-4e49-8f03-ec052fb52046/volumes"
Sep 08 20:26:51.002926 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:51.002842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cl6hs\" (UID: \"d2ed3c01-5a4d-475b-84f3-97c7969c52da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:51.005676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:51.005648    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d2ed3c01-5a4d-475b-84f3-97c7969c52da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cl6hs\" (UID: \"d2ed3c01-5a4d-475b-84f3-97c7969c52da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:51.265701 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:51.265624    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-h8wxl\""
Sep 08 20:26:51.273434 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:51.273397    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"
Sep 08 20:26:52.200032 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:52.200000    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:52.301512 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:52.301483    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:52.466999 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:52.466898    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs"]
Sep 08 20:26:52.470299 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:26:52.470268    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2ed3c01_5a4d_475b_84f3_97c7969c52da.slice/crio-45ef76ad1b32063738cdc818b261cee51c2a59bf3e68dd73f0b7bfc9a7ea9eb9 WatchSource:0}: Error finding container 45ef76ad1b32063738cdc818b261cee51c2a59bf3e68dd73f0b7bfc9a7ea9eb9: Status 404 returned error can't find the container with id 45ef76ad1b32063738cdc818b261cee51c2a59bf3e68dd73f0b7bfc9a7ea9eb9
Sep 08 20:26:53.288515 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:53.288474    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d" event={"ID":"d07d64de-c856-4b22-82c7-fe1f891d6c8a","Type":"ContainerStarted","Data":"c5f3b92256dd9993a44e6a871b2d997c1706070519e1489a5879cfa17ab7d127"}
Sep 08 20:26:53.288988 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:53.288674    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d"
Sep 08 20:26:53.290129 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:53.290104    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs" event={"ID":"d2ed3c01-5a4d-475b-84f3-97c7969c52da","Type":"ContainerStarted","Data":"45ef76ad1b32063738cdc818b261cee51c2a59bf3e68dd73f0b7bfc9a7ea9eb9"}
Sep 08 20:26:53.290780 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:53.290755    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d"
Sep 08 20:26:53.306007 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:53.305936    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c75987765-z862d" podStartSLOduration=2.044199996 podStartE2EDuration="6.30591917s" totalImagesPullingTime="4.261719174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:53.305117692 +0000 UTC m=+86.852367174" watchObservedRunningTime="2026-09-08 20:26:53.30591917 +0000 UTC m=+86.853168653"
Sep 08 20:26:53.980407 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:53.980362    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:54.081269 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:54.081233    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:54.180421 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:54.180395    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:54.296388 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:54.296285    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs" event={"ID":"d2ed3c01-5a4d-475b-84f3-97c7969c52da","Type":"ContainerStarted","Data":"f3cfa4f1a1108e2dd0aaaa5b1c0777b4eb29afb472a4b54f1d3d527b46e076d7"}
Sep 08 20:26:54.319252 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:54.319192    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cl6hs" podStartSLOduration=33.621756994 podStartE2EDuration="35.319175432s" totalImagesPullingTime="1.697418438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:54.318359067 +0000 UTC m=+87.865608545" watchObservedRunningTime="2026-09-08 20:26:54.319175432 +0000 UTC m=+87.866424912"
Sep 08 20:26:54.420382 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:54.420351    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:54.518874 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:54.518835    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:54.654490 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:54.654461    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:54.664168 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:54.664143    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qlx76"]
Sep 08 20:26:54.696864 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:54.696828    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qlx76"]
Sep 08 20:26:54.697043 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:54.696975    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qlx76"
Sep 08 20:26:54.699288 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:54.699267    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 08 20:26:54.699652 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:54.699634    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-wzkfh\""
Sep 08 20:26:54.781001 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:54.780950    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:54.839296 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:54.839261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/54f2231d-9338-406a-bb23-e3236edd4f60-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-qlx76\" (UID: \"54f2231d-9338-406a-bb23-e3236edd4f60\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qlx76"
Sep 08 20:26:54.845015 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:54.844991    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:54.939855 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:54.939779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/54f2231d-9338-406a-bb23-e3236edd4f60-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-qlx76\" (UID: \"54f2231d-9338-406a-bb23-e3236edd4f60\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qlx76"
Sep 08 20:26:54.940009 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:54.939934    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 08 20:26:54.940061 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:26:54.940018    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f2231d-9338-406a-bb23-e3236edd4f60-tls-certificates podName:54f2231d-9338-406a-bb23-e3236edd4f60 nodeName:}" failed. No retries permitted until 2026-09-08 20:26:55.440001501 +0000 UTC m=+88.987250963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/54f2231d-9338-406a-bb23-e3236edd4f60-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-qlx76" (UID: "54f2231d-9338-406a-bb23-e3236edd4f60") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 08 20:26:55.272512 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:55.272415    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:55.444250 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:55.444206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/54f2231d-9338-406a-bb23-e3236edd4f60-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-qlx76\" (UID: \"54f2231d-9338-406a-bb23-e3236edd4f60\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qlx76"
Sep 08 20:26:55.446629 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:55.446598    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/54f2231d-9338-406a-bb23-e3236edd4f60-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-qlx76\" (UID: \"54f2231d-9338-406a-bb23-e3236edd4f60\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qlx76"
Sep 08 20:26:55.605881 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:55.605783    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qlx76"
Sep 08 20:26:55.893448 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:55.893417    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qlx76"]
Sep 08 20:26:55.896702 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:26:55.896663    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54f2231d_9338_406a_bb23_e3236edd4f60.slice/crio-de1605fccc1dd7b33012e1137285d1a5470cf1f7af05e2ebfde381967d5b2029 WatchSource:0}: Error finding container de1605fccc1dd7b33012e1137285d1a5470cf1f7af05e2ebfde381967d5b2029: Status 404 returned error can't find the container with id de1605fccc1dd7b33012e1137285d1a5470cf1f7af05e2ebfde381967d5b2029
Sep 08 20:26:56.307871 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:56.307788    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qlx76" event={"ID":"54f2231d-9338-406a-bb23-e3236edd4f60","Type":"ContainerStarted","Data":"de1605fccc1dd7b33012e1137285d1a5470cf1f7af05e2ebfde381967d5b2029"}
Sep 08 20:26:57.361736 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:57.361693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:57.364225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:57.364202    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4d58a5e-85e0-4276-a8d5-869820c7f439-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnccs\" (UID: \"d4d58a5e-85e0-4276-a8d5-869820c7f439\") " pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:57.626687 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:57.626648    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4gd8m\""
Sep 08 20:26:57.635357 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:57.635330    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hnccs"
Sep 08 20:26:57.905583 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:57.905499    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hnccs"]
Sep 08 20:26:57.909636 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:26:57.909600    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4d58a5e_85e0_4276_a8d5_869820c7f439.slice/crio-4b81d01b767dcf18e273c966c6390fc6cee02fd0e49ffcde64fcbbaba4507e00 WatchSource:0}: Error finding container 4b81d01b767dcf18e273c966c6390fc6cee02fd0e49ffcde64fcbbaba4507e00: Status 404 returned error can't find the container with id 4b81d01b767dcf18e273c966c6390fc6cee02fd0e49ffcde64fcbbaba4507e00
Sep 08 20:26:58.317487 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:58.317405    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnccs" event={"ID":"d4d58a5e-85e0-4276-a8d5-869820c7f439","Type":"ContainerStarted","Data":"4660f99800abf59b9b58863b51c4bb0025d0294e89c53227dc9c024a075b0a87"}
Sep 08 20:26:58.317487 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:58.317465    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnccs" event={"ID":"d4d58a5e-85e0-4276-a8d5-869820c7f439","Type":"ContainerStarted","Data":"4b81d01b767dcf18e273c966c6390fc6cee02fd0e49ffcde64fcbbaba4507e00"}
Sep 08 20:26:58.318863 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:58.318840    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qlx76" event={"ID":"54f2231d-9338-406a-bb23-e3236edd4f60","Type":"ContainerStarted","Data":"378652c00e839778eb8beab3df43c9ea4d8025c79991e100fd9c07984aa9ad29"}
Sep 08 20:26:58.319073 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:58.319056    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qlx76"
Sep 08 20:26:58.323574 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:58.323542    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qlx76"
Sep 08 20:26:58.334440 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:58.334390    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-qlx76" podStartSLOduration=2.801412831 podStartE2EDuration="4.334374279s" totalImagesPullingTime="1.532961448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:26:58.333653427 +0000 UTC m=+91.880902907" watchObservedRunningTime="2026-09-08 20:26:58.334374279 +0000 UTC m=+91.881623760"
Sep 08 20:26:58.684191 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:58.684158    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:58.765088 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:58.765052    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:59.255227 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:59.255188    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:26:59.325145 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:26:59.325105    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnccs" event={"ID":"d4d58a5e-85e0-4276-a8d5-869820c7f439","Type":"ContainerStarted","Data":"1f46237f38e1c8306d5c4847ad6ee5e234a45a182d48a5c39f8677661e1a44da"}
Sep 08 20:27:00.330567 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:00.330531    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnccs" event={"ID":"d4d58a5e-85e0-4276-a8d5-869820c7f439","Type":"ContainerStarted","Data":"7898e908d63f623c695bce29f3e57c6eba55818d68684f645756f48a7db9e5a7"}
Sep 08 20:27:00.351943 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:00.351882    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hnccs" podStartSLOduration=33.253947741 podStartE2EDuration="35.351865207s" totalImagesPullingTime="2.097917466s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 20:26:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:00.350990994 +0000 UTC m=+93.898240476" watchObservedRunningTime="2026-09-08 20:27:00.351865207 +0000 UTC m=+93.899114688"
Sep 08 20:27:00.711138 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:00.711104    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:00.767615 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:00.767589    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:01.387052 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.387014    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f9b546bb-rqngb"]
Sep 08 20:27:01.389208 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.389190    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.391354 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.391325    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 08 20:27:01.391878 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.391855    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zlxsw\""
Sep 08 20:27:01.392066 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.392042    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 08 20:27:01.392066 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.392070    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 08 20:27:01.392226 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.391877    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 08 20:27:01.392226 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.391925    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 08 20:27:01.392226 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.391914    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 08 20:27:01.392226 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.392016    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 08 20:27:01.396179 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.396163    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 08 20:27:01.400200 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.400176    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f9b546bb-rqngb"]
Sep 08 20:27:01.461283 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.461248    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:01.496515 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.496476    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-trusted-ca-bundle\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.496515 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.496510    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9xzd\" (UniqueName: \"kubernetes.io/projected/5f85dbad-2508-4c7e-aaae-c496346d6086-kube-api-access-l9xzd\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.496721 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.496536    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-oauth-serving-cert\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.496721 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.496570    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f85dbad-2508-4c7e-aaae-c496346d6086-console-serving-cert\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.496721 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.496665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f85dbad-2508-4c7e-aaae-c496346d6086-console-oauth-config\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.496721 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.496702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-console-config\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.496870 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.496731    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-service-ca\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.518827 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.518800    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:01.598075 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.598038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f85dbad-2508-4c7e-aaae-c496346d6086-console-oauth-config\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.598075 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.598075    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-console-config\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.598243 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.598094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-service-ca\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.598243 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.598127    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-trusted-ca-bundle\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.598243 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.598151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9xzd\" (UniqueName: \"kubernetes.io/projected/5f85dbad-2508-4c7e-aaae-c496346d6086-kube-api-access-l9xzd\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.598243 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.598188    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-oauth-serving-cert\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.598243 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.598218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f85dbad-2508-4c7e-aaae-c496346d6086-console-serving-cert\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.598976 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.598924    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-service-ca\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.599083 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.599023    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-console-config\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.599129 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.599098    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-oauth-serving-cert\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.599175 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.599162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-trusted-ca-bundle\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.600626 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.600606    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f85dbad-2508-4c7e-aaae-c496346d6086-console-serving-cert\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.600715 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.600657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f85dbad-2508-4c7e-aaae-c496346d6086-console-oauth-config\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.606216 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.606195    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9xzd\" (UniqueName: \"kubernetes.io/projected/5f85dbad-2508-4c7e-aaae-c496346d6086-kube-api-access-l9xzd\") pod \"console-6f9b546bb-rqngb\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") " pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.679515 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.679430    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:01.699139 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.699113    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:01.968299 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:01.968177    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f9b546bb-rqngb"]
Sep 08 20:27:01.970864 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:27:01.970830    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f85dbad_2508_4c7e_aaae_c496346d6086.slice/crio-1469ddeb4f7ae3b133b8b63d3e0a42aa484cff0a0b1afba9749e8e94f7704dcb WatchSource:0}: Error finding container 1469ddeb4f7ae3b133b8b63d3e0a42aa484cff0a0b1afba9749e8e94f7704dcb: Status 404 returned error can't find the container with id 1469ddeb4f7ae3b133b8b63d3e0a42aa484cff0a0b1afba9749e8e94f7704dcb
Sep 08 20:27:02.338758 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:02.338673    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9b546bb-rqngb" event={"ID":"5f85dbad-2508-4c7e-aaae-c496346d6086","Type":"ContainerStarted","Data":"1469ddeb4f7ae3b133b8b63d3e0a42aa484cff0a0b1afba9749e8e94f7704dcb"}
Sep 08 20:27:04.815193 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:04.815147    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:04.826253 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:04.826217    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:27:04.829055 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:04.829024    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34b69620-f3c4-471b-85d5-887e22e90910-metrics-tls\") pod \"dns-default-75k8c\" (UID: \"34b69620-f3c4-471b-85d5-887e22e90910\") " pod="openshift-dns/dns-default-75k8c"
Sep 08 20:27:04.918897 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:04.918815    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:05.080174 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:05.080101    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vbwpr\""
Sep 08 20:27:05.088195 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:05.088158    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-75k8c"
Sep 08 20:27:05.215585 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:05.215547    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vjhfp"
Sep 08 20:27:05.640728 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:05.640694    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-75k8c"]
Sep 08 20:27:05.643473 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:27:05.643433    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b69620_f3c4_471b_85d5_887e22e90910.slice/crio-353f332415279c0aa4e59d1f42b8893a6ae6c0d5e144f8c53745fec5b4228d88 WatchSource:0}: Error finding container 353f332415279c0aa4e59d1f42b8893a6ae6c0d5e144f8c53745fec5b4228d88: Status 404 returned error can't find the container with id 353f332415279c0aa4e59d1f42b8893a6ae6c0d5e144f8c53745fec5b4228d88
Sep 08 20:27:06.355325 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:06.355287    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9b546bb-rqngb" event={"ID":"5f85dbad-2508-4c7e-aaae-c496346d6086","Type":"ContainerStarted","Data":"874a3b344ab42e7ce4099a3b30f92dbf82e4ff9c72e891ccb06ab30aa40e885b"}
Sep 08 20:27:06.356600 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:06.356571    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-75k8c" event={"ID":"34b69620-f3c4-471b-85d5-887e22e90910","Type":"ContainerStarted","Data":"353f332415279c0aa4e59d1f42b8893a6ae6c0d5e144f8c53745fec5b4228d88"}
Sep 08 20:27:06.625336 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:06.625249    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:06.724334 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:06.724301    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:07.866636 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:07.866599    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:08.366896 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:08.366854    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-75k8c" event={"ID":"34b69620-f3c4-471b-85d5-887e22e90910","Type":"ContainerStarted","Data":"158931d6234fa532d6b06c683143edcb4fd6505ea16bd220061fa8cdf518cf5f"}
Sep 08 20:27:08.367126 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:08.366994    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-75k8c" event={"ID":"34b69620-f3c4-471b-85d5-887e22e90910","Type":"ContainerStarted","Data":"3cf7e2fce8ca0729ca7fa681419be3dc9dc253c44141a54345bd608efdf60131"}
Sep 08 20:27:08.367126 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:08.367020    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-75k8c"
Sep 08 20:27:08.388659 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:08.388604    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f9b546bb-rqngb" podStartSLOduration=3.963115226 podStartE2EDuration="7.388587387s" totalImagesPullingTime="3.425472161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:06.373076366 +0000 UTC m=+99.920325848" watchObservedRunningTime="2026-09-08 20:27:08.388587387 +0000 UTC m=+101.935836868"
Sep 08 20:27:08.388884 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:08.388864    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-75k8c" podStartSLOduration=66.738185007 podStartE2EDuration="1m8.388857934s" totalImagesPullingTime="1.650672927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:26:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:08.388268275 +0000 UTC m=+101.935517758" watchObservedRunningTime="2026-09-08 20:27:08.388857934 +0000 UTC m=+101.936107415"
Sep 08 20:27:09.842631 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:09.842599    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:10.155596 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.155553    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:10.232479 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.232445    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6sfwv"]
Sep 08 20:27:10.255548 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.255519    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:10.264916 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.264887    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.267055 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.267029    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nwv9r\""
Sep 08 20:27:10.267202 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.267063    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 08 20:27:10.267202 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.267084    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 08 20:27:10.267202 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.267096    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 08 20:27:10.267793 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.267776    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 08 20:27:10.273170 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.273151    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca31dae6-e536-465d-83e6-6a0f18ecadbf-root\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.273247 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.273212    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-textfile\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.273247 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.273239    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-tls\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.273317 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.273268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca31dae6-e536-465d-83e6-6a0f18ecadbf-metrics-client-ca\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.273317 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.273287    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v29h\" (UniqueName: \"kubernetes.io/projected/ca31dae6-e536-465d-83e6-6a0f18ecadbf-kube-api-access-6v29h\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.273375 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.273332    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-wtmp\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.273375 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.273362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.273482 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.273463    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca31dae6-e536-465d-83e6-6a0f18ecadbf-sys\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.273513 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.273495    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.281229 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.281204    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-79f5ff44f6-sgnmg"
Sep 08 20:27:10.336244 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.336216    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:10.374262 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.374224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.374443 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.374299    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca31dae6-e536-465d-83e6-6a0f18ecadbf-root\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.374443 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.374362    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca31dae6-e536-465d-83e6-6a0f18ecadbf-root\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.374558 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.374493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-textfile\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.374558 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.374529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-tls\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.374655 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.374610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca31dae6-e536-465d-83e6-6a0f18ecadbf-metrics-client-ca\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.374709 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.374662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v29h\" (UniqueName: \"kubernetes.io/projected/ca31dae6-e536-465d-83e6-6a0f18ecadbf-kube-api-access-6v29h\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.374709 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.374693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-wtmp\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.374796 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.374720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.374796 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.374764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca31dae6-e536-465d-83e6-6a0f18ecadbf-sys\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.374893 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.374805    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-textfile\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.374893 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.374860    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca31dae6-e536-465d-83e6-6a0f18ecadbf-sys\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.375011 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.374890    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-wtmp\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.375011 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:27:10.374916    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 08 20:27:10.375102 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:27:10.375013    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-tls podName:ca31dae6-e536-465d-83e6-6a0f18ecadbf nodeName:}" failed. No retries permitted until 2026-09-08 20:27:10.874992941 +0000 UTC m=+104.422242416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-tls") pod "node-exporter-6sfwv" (UID: "ca31dae6-e536-465d-83e6-6a0f18ecadbf") : secret "node-exporter-tls" not found
Sep 08 20:27:10.375267 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.375243    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca31dae6-e536-465d-83e6-6a0f18ecadbf-metrics-client-ca\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.375322 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.375304    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.376684 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.376667    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.385040 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.385009    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v29h\" (UniqueName: \"kubernetes.io/projected/ca31dae6-e536-465d-83e6-6a0f18ecadbf-kube-api-access-6v29h\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.433338 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.433254    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:10.533659 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.533614    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:10.634140 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.634107    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:10.816649 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.816574    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:10.878969 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.878919    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-tls\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:10.881218 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:10.881200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca31dae6-e536-465d-83e6-6a0f18ecadbf-node-exporter-tls\") pod \"node-exporter-6sfwv\" (UID: \"ca31dae6-e536-465d-83e6-6a0f18ecadbf\") " pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:11.065578 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:11.065527    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:11.151164 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:11.151131    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:11.174276 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:11.174243    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6sfwv"
Sep 08 20:27:11.185117 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:27:11.185092    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca31dae6_e536_465d_83e6_6a0f18ecadbf.slice/crio-d440c33a8f27c26978f21fececb80d383258f703bd3e839069f8e1bef1ad0229 WatchSource:0}: Error finding container d440c33a8f27c26978f21fececb80d383258f703bd3e839069f8e1bef1ad0229: Status 404 returned error can't find the container with id d440c33a8f27c26978f21fececb80d383258f703bd3e839069f8e1bef1ad0229
Sep 08 20:27:11.300490 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:11.300452    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:11.378932 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:11.378893    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sfwv" event={"ID":"ca31dae6-e536-465d-83e6-6a0f18ecadbf","Type":"ContainerStarted","Data":"d440c33a8f27c26978f21fececb80d383258f703bd3e839069f8e1bef1ad0229"}
Sep 08 20:27:11.484371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:11.484277    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:11.699782 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:11.699740    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:11.700006 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:11.699817    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:11.705268 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:11.705226    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:11.798902 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:11.798819    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:11.999633 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:11.999592    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:12.089502 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.089478    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"]
Sep 08 20:27:12.092044 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.092021    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.094395 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.094370    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 08 20:27:12.094395 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.094389    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-47t6v\""
Sep 08 20:27:12.094548 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.094416    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 08 20:27:12.094548 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.094440    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 08 20:27:12.094548 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.094375    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 08 20:27:12.094702 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.094554    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5a2d712q6q33l\""
Sep 08 20:27:12.094751 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.094726    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 08 20:27:12.104299 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.104244    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:12.104728 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.104708    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"]
Sep 08 20:27:12.189401 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.189363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.189401 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.189405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-metrics-client-ca\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.189601 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.189430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.189601 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.189503    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-grpc-tls\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.189601 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.189530    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-thanos-querier-tls\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.189601 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.189551    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.189601 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.189579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q72l\" (UniqueName: \"kubernetes.io/projected/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-kube-api-access-5q72l\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.189749 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.189661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.290302 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.290266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.290480 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.290327    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.290480 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.290352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-metrics-client-ca\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.290480 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.290378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.291064 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.290623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-grpc-tls\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.291064 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.290673    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-thanos-querier-tls\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.291064 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.290713    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.291064 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.290752    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q72l\" (UniqueName: \"kubernetes.io/projected/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-kube-api-access-5q72l\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.291401 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.291200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-metrics-client-ca\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.293357 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.293257    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.293357 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.293315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-grpc-tls\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.293529 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.293389    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.293529 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.293415    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.293529 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.293441    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-thanos-querier-tls\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.293529 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.293510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.298241 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.298219    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q72l\" (UniqueName: \"kubernetes.io/projected/4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5-kube-api-access-5q72l\") pod \"thanos-querier-7c78bc8cf7-z77f8\" (UID: \"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5\") " pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.387294 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.387260    2580 generic.go:356] "Generic (PLEG): container finished" podID="ca31dae6-e536-465d-83e6-6a0f18ecadbf" containerID="bccc63a64ef6cfe9071cfd6f35944863bf972f040cac341ab37e062fd14fe09b" exitCode=0
Sep 08 20:27:12.387474 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.387350    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sfwv" event={"ID":"ca31dae6-e536-465d-83e6-6a0f18ecadbf","Type":"ContainerDied","Data":"bccc63a64ef6cfe9071cfd6f35944863bf972f040cac341ab37e062fd14fe09b"}
Sep 08 20:27:12.391528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.391499    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:12.402017 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.401989    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:12.684160 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:12.684071    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"]
Sep 08 20:27:12.687663 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:27:12.687636    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f763e45_1fe8_42f9_aa7b_6cf79ec7dae5.slice/crio-6ac6fe531be2414997484dc8017b6a5ed28f4a0fcc8958c4cac10d6f528e094f WatchSource:0}: Error finding container 6ac6fe531be2414997484dc8017b6a5ed28f4a0fcc8958c4cac10d6f528e094f: Status 404 returned error can't find the container with id 6ac6fe531be2414997484dc8017b6a5ed28f4a0fcc8958c4cac10d6f528e094f
Sep 08 20:27:13.031596 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:13.031509    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:13.129676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:13.129640    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:13.228317 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:13.228271    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:13.331830 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:13.331748    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:13.392009 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:13.391949    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8" event={"ID":"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5","Type":"ContainerStarted","Data":"6ac6fe531be2414997484dc8017b6a5ed28f4a0fcc8958c4cac10d6f528e094f"}
Sep 08 20:27:13.394078 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:13.394054    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sfwv" event={"ID":"ca31dae6-e536-465d-83e6-6a0f18ecadbf","Type":"ContainerStarted","Data":"cfad94c8a060fce959496d47adb28d624910245396d48c62d50ce1d30b8f13d1"}
Sep 08 20:27:13.394197 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:13.394105    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sfwv" event={"ID":"ca31dae6-e536-465d-83e6-6a0f18ecadbf","Type":"ContainerStarted","Data":"cd15ee9b6f8e64dc157271bd6f76b9c14a71c72b2095e480f6db5604db775ee1"}
Sep 08 20:27:13.413076 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:13.413017    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6sfwv" podStartSLOduration=2.5051693889999997 podStartE2EDuration="3.412997433s" totalImagesPullingTime="842.76003ms" totalInitContainerRuntime="65.068014ms" isStatefulPod=true podCreationTimestamp="2026-09-08 20:27:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:13.412381242 +0000 UTC m=+106.959630723" watchObservedRunningTime="2026-09-08 20:27:13.412997433 +0000 UTC m=+106.960246915"
Sep 08 20:27:13.678651 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:13.678614    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:13.732058 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:13.732024    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:13.812576 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:13.812536    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:13.913269 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:13.913234    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:14.580034 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.580007    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"]
Sep 08 20:27:14.582018 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.581997    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.584746 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.584723    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 08 20:27:14.584875 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.584855    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 08 20:27:14.585084 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.584864    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 08 20:27:14.585084 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.584984    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9khmc\""
Sep 08 20:27:14.585084 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.585062    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 08 20:27:14.585296 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.585280    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dhipmj6d1972s\""
Sep 08 20:27:14.594139 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.594118    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"]
Sep 08 20:27:14.612048 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.612019    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/aca1d922-fe5e-462d-a550-e29a90c25ae6-metrics-server-audit-profiles\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.612136 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.612065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-855td\" (UniqueName: \"kubernetes.io/projected/aca1d922-fe5e-462d-a550-e29a90c25ae6-kube-api-access-855td\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.612176 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.612161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aca1d922-fe5e-462d-a550-e29a90c25ae6-client-ca-bundle\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.612214 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.612195    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/aca1d922-fe5e-462d-a550-e29a90c25ae6-secret-metrics-server-tls\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.612270 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.612255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aca1d922-fe5e-462d-a550-e29a90c25ae6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.612321 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.612312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/aca1d922-fe5e-462d-a550-e29a90c25ae6-audit-log\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.612357 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.612336    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/aca1d922-fe5e-462d-a550-e29a90c25ae6-secret-metrics-server-client-certs\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.713127 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.713100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/aca1d922-fe5e-462d-a550-e29a90c25ae6-audit-log\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.713276 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.713149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/aca1d922-fe5e-462d-a550-e29a90c25ae6-secret-metrics-server-client-certs\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.713350 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.713327    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/aca1d922-fe5e-462d-a550-e29a90c25ae6-metrics-server-audit-profiles\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.713402 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.713366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-855td\" (UniqueName: \"kubernetes.io/projected/aca1d922-fe5e-462d-a550-e29a90c25ae6-kube-api-access-855td\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.713461 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.713401    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aca1d922-fe5e-462d-a550-e29a90c25ae6-client-ca-bundle\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.713461 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.713435    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/aca1d922-fe5e-462d-a550-e29a90c25ae6-secret-metrics-server-tls\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.713567 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.713500    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aca1d922-fe5e-462d-a550-e29a90c25ae6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.713625 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.713574    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/aca1d922-fe5e-462d-a550-e29a90c25ae6-audit-log\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.714100 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.714077    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aca1d922-fe5e-462d-a550-e29a90c25ae6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.714409 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.714387    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/aca1d922-fe5e-462d-a550-e29a90c25ae6-metrics-server-audit-profiles\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.715871 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.715846    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aca1d922-fe5e-462d-a550-e29a90c25ae6-client-ca-bundle\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.716140 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.716120    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/aca1d922-fe5e-462d-a550-e29a90c25ae6-secret-metrics-server-client-certs\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.716237 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.716215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/aca1d922-fe5e-462d-a550-e29a90c25ae6-secret-metrics-server-tls\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.721825 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.721801    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-855td\" (UniqueName: \"kubernetes.io/projected/aca1d922-fe5e-462d-a550-e29a90c25ae6-kube-api-access-855td\") pod \"metrics-server-7b9ff5f76f-f5nhx\" (UID: \"aca1d922-fe5e-462d-a550-e29a90c25ae6\") " pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.856396 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.856308    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:14.866549 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.866522    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-wgmpq"]
Sep 08 20:27:14.868746 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.868730    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wgmpq"
Sep 08 20:27:14.870903 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.870880    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 08 20:27:14.871033 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.871011    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-72j7k\""
Sep 08 20:27:14.877583 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.877561    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-wgmpq"]
Sep 08 20:27:14.910940 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.910914    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:14.915388 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.915359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d30e5889-153d-4d96-bcd2-9b448b4dbcd1-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-wgmpq\" (UID: \"d30e5889-153d-4d96-bcd2-9b448b4dbcd1\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wgmpq"
Sep 08 20:27:14.915497 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.915477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxrd8\" (UniqueName: \"kubernetes.io/projected/d30e5889-153d-4d96-bcd2-9b448b4dbcd1-kube-api-access-qxrd8\") pod \"monitoring-plugin-85bbbbbf4-wgmpq\" (UID: \"d30e5889-153d-4d96-bcd2-9b448b4dbcd1\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wgmpq"
Sep 08 20:27:14.957077 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:14.957048    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:15.016648 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.016613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d30e5889-153d-4d96-bcd2-9b448b4dbcd1-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-wgmpq\" (UID: \"d30e5889-153d-4d96-bcd2-9b448b4dbcd1\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wgmpq"
Sep 08 20:27:15.016825 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.016677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxrd8\" (UniqueName: \"kubernetes.io/projected/d30e5889-153d-4d96-bcd2-9b448b4dbcd1-kube-api-access-qxrd8\") pod \"monitoring-plugin-85bbbbbf4-wgmpq\" (UID: \"d30e5889-153d-4d96-bcd2-9b448b4dbcd1\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wgmpq"
Sep 08 20:27:15.019019 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.018994    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d30e5889-153d-4d96-bcd2-9b448b4dbcd1-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-wgmpq\" (UID: \"d30e5889-153d-4d96-bcd2-9b448b4dbcd1\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wgmpq"
Sep 08 20:27:15.029250 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.029223    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxrd8\" (UniqueName: \"kubernetes.io/projected/d30e5889-153d-4d96-bcd2-9b448b4dbcd1-kube-api-access-qxrd8\") pod \"monitoring-plugin-85bbbbbf4-wgmpq\" (UID: \"d30e5889-153d-4d96-bcd2-9b448b4dbcd1\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wgmpq"
Sep 08 20:27:15.029747 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.029726    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:15.177707 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.177675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wgmpq"
Sep 08 20:27:15.179796 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.179764    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"]
Sep 08 20:27:15.182727 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:27:15.182696    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaca1d922_fe5e_462d_a550_e29a90c25ae6.slice/crio-aedeb7cab88ba9d5ce5882021a41e42e5d42a501788734e9a84e751d2c16b6bf WatchSource:0}: Error finding container aedeb7cab88ba9d5ce5882021a41e42e5d42a501788734e9a84e751d2c16b6bf: Status 404 returned error can't find the container with id aedeb7cab88ba9d5ce5882021a41e42e5d42a501788734e9a84e751d2c16b6bf
Sep 08 20:27:15.402970 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.402933    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8" event={"ID":"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5","Type":"ContainerStarted","Data":"283e87182d11c08f9f5578880ae39807d0d4f65863711a33c2cdaa7b374eab2b"}
Sep 08 20:27:15.403159 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.403040    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8" event={"ID":"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5","Type":"ContainerStarted","Data":"02daf0fec38f4091465807436d04a4f2b23404e0b758775666d4822809a5b8d5"}
Sep 08 20:27:15.403159 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.403086    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8" event={"ID":"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5","Type":"ContainerStarted","Data":"037e778729646de0c780a6752e0929c9e184d47c5e5c2a1eb6a5a1f15ceb0fdc"}
Sep 08 20:27:15.404150 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.404128    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx" event={"ID":"aca1d922-fe5e-462d-a550-e29a90c25ae6","Type":"ContainerStarted","Data":"aedeb7cab88ba9d5ce5882021a41e42e5d42a501788734e9a84e751d2c16b6bf"}
Sep 08 20:27:15.447208 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.447183    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-wgmpq"]
Sep 08 20:27:15.450030 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:27:15.449990    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd30e5889_153d_4d96_bcd2_9b448b4dbcd1.slice/crio-8e77b9e63f5ec100eb77b61c4206b7e34a181da9cb54043cbfb73b634394cfac WatchSource:0}: Error finding container 8e77b9e63f5ec100eb77b61c4206b7e34a181da9cb54043cbfb73b634394cfac: Status 404 returned error can't find the container with id 8e77b9e63f5ec100eb77b61c4206b7e34a181da9cb54043cbfb73b634394cfac
Sep 08 20:27:15.804386 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.804353    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-864f669fc-wdh6q"]
Sep 08 20:27:15.806932 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.806911    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:15.816533 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.816509    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-864f669fc-wdh6q"]
Sep 08 20:27:15.923564 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.923536    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1298d0be-1d59-45f7-b4b0-c02254abc260-console-oauth-config\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:15.923706 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.923588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8nbf\" (UniqueName: \"kubernetes.io/projected/1298d0be-1d59-45f7-b4b0-c02254abc260-kube-api-access-q8nbf\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:15.923706 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.923649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-trusted-ca-bundle\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:15.923706 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.923697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-service-ca\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:15.923866 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.923790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-oauth-serving-cert\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:15.923866 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.923831    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1298d0be-1d59-45f7-b4b0-c02254abc260-console-serving-cert\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:15.923948 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:15.923917    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-console-config\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:16.024508 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.024471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-console-config\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:16.024660 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.024543    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1298d0be-1d59-45f7-b4b0-c02254abc260-console-oauth-config\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:16.024660 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.024587    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8nbf\" (UniqueName: \"kubernetes.io/projected/1298d0be-1d59-45f7-b4b0-c02254abc260-kube-api-access-q8nbf\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:16.024660 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.024615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-trusted-ca-bundle\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:16.024660 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.024644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-service-ca\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:16.024809 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.024701    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-oauth-serving-cert\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:16.024809 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.024729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1298d0be-1d59-45f7-b4b0-c02254abc260-console-serving-cert\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:16.024996 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.024479    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:16.025448 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.025313    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-console-config\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:16.025715 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.025693    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-service-ca\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:16.026235 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.026213    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-oauth-serving-cert\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:16.026509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.026456    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-trusted-ca-bundle\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:16.027884 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.027855    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1298d0be-1d59-45f7-b4b0-c02254abc260-console-serving-cert\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:16.028270 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.028229    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1298d0be-1d59-45f7-b4b0-c02254abc260-console-oauth-config\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:16.039680 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.039597    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8nbf\" (UniqueName: \"kubernetes.io/projected/1298d0be-1d59-45f7-b4b0-c02254abc260-kube-api-access-q8nbf\") pod \"console-864f669fc-wdh6q\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") " pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:16.125710 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.125671    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:16.134569 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.134528    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:16.299564 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.299479    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:16.418720 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.418108    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-864f669fc-wdh6q"]
Sep 08 20:27:16.419362 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.419328    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8" event={"ID":"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5","Type":"ContainerStarted","Data":"6206e820bfa4c75a92dffea7620c8a6f996b10d38d98aea363343c097dd0d324"}
Sep 08 20:27:16.419474 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.419402    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8" event={"ID":"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5","Type":"ContainerStarted","Data":"97dfd503fd3970479d69b487fd7de3e1b61d6d19d397327113756ee62fc2a590"}
Sep 08 20:27:16.419474 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.419444    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8" event={"ID":"4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5","Type":"ContainerStarted","Data":"58b47e7a3851326e16cdd6977e9538b53c5afb5f113d2235d86be6cd22a4fdfe"}
Sep 08 20:27:16.419608 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.419489    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:16.420932 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.420906    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wgmpq" event={"ID":"d30e5889-153d-4d96-bcd2-9b448b4dbcd1","Type":"ContainerStarted","Data":"8e77b9e63f5ec100eb77b61c4206b7e34a181da9cb54043cbfb73b634394cfac"}
Sep 08 20:27:16.443191 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.443137    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8" podStartSLOduration=1.534141301 podStartE2EDuration="4.443119771s" totalImagesPullingTime="2.90897847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:27:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:16.441122328 +0000 UTC m=+109.988371809" watchObservedRunningTime="2026-09-08 20:27:16.443119771 +0000 UTC m=+109.990369252"
Sep 08 20:27:16.744709 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:27:16.744655    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1298d0be_1d59_45f7_b4b0_c02254abc260.slice/crio-b26f04d456583387a034e2f08968c0ed3facb850f152cbcd5a6007c0a629bca1 WatchSource:0}: Error finding container b26f04d456583387a034e2f08968c0ed3facb850f152cbcd5a6007c0a629bca1: Status 404 returned error can't find the container with id b26f04d456583387a034e2f08968c0ed3facb850f152cbcd5a6007c0a629bca1
Sep 08 20:27:16.989485 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:16.989442    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:17.428038 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:17.427915    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864f669fc-wdh6q" event={"ID":"1298d0be-1d59-45f7-b4b0-c02254abc260","Type":"ContainerStarted","Data":"d0e89d295137cebb56131598e0e5d405fa8be310227825e9f665beeb68fbd656"}
Sep 08 20:27:17.428038 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:17.427987    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864f669fc-wdh6q" event={"ID":"1298d0be-1d59-45f7-b4b0-c02254abc260","Type":"ContainerStarted","Data":"b26f04d456583387a034e2f08968c0ed3facb850f152cbcd5a6007c0a629bca1"}
Sep 08 20:27:17.429613 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:17.429586    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wgmpq" event={"ID":"d30e5889-153d-4d96-bcd2-9b448b4dbcd1","Type":"ContainerStarted","Data":"a7056255368d3db59bc39568179b41130485d3e1dce20a6b08a3b5c37031b7ef"}
Sep 08 20:27:17.429748 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:17.429641    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wgmpq"
Sep 08 20:27:17.431259 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:17.431227    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx" event={"ID":"aca1d922-fe5e-462d-a550-e29a90c25ae6","Type":"ContainerStarted","Data":"4e300ca56f0992ef2596b32a360242253ffe9bc1616b392974bbb709ff369a33"}
Sep 08 20:27:17.434897 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:17.434876    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wgmpq"
Sep 08 20:27:17.444421 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:17.444373    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-864f669fc-wdh6q" podStartSLOduration=2.44436051 podStartE2EDuration="2.44436051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:27:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:17.443550261 +0000 UTC m=+110.990799744" watchObservedRunningTime="2026-09-08 20:27:17.44436051 +0000 UTC m=+110.991609991"
Sep 08 20:27:17.459676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:17.459618    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx" podStartSLOduration=1.480681447 podStartE2EDuration="3.459597714s" totalImagesPullingTime="1.978916267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:17.458274751 +0000 UTC m=+111.005524232" watchObservedRunningTime="2026-09-08 20:27:17.459597714 +0000 UTC m=+111.006847197"
Sep 08 20:27:17.475792 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:17.475741    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-wgmpq" podStartSLOduration=1.766424083 podStartE2EDuration="3.475718571s" totalImagesPullingTime="1.709294488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:17.473764767 +0000 UTC m=+111.021014248" watchObservedRunningTime="2026-09-08 20:27:17.475718571 +0000 UTC m=+111.022968054"
Sep 08 20:27:18.372929 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:18.372900    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-75k8c"
Sep 08 20:27:22.012757 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:22.012724    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:22.113263 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:22.113229    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:22.212715 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:22.212680    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:22.439459 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:22.439423    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7c78bc8cf7-z77f8"
Sep 08 20:27:26.135543 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:26.135500    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:26.135937 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:26.135586    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:26.138311 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:26.138280    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:26.140490 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:26.140469    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:26.238697 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:26.238666    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:26.471198 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:26.471111    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:27:26.520004 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:26.519975    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f9b546bb-rqngb"]
Sep 08 20:27:26.695859 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:26.695823    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:26.795904 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:26.795812    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:26.925944 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:26.925901    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:26.979941 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:26.979906    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:27.487616 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:27.487579    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:27.548890 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:27.548859    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:29.478920 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:29.478889    2580 generic.go:356] "Generic (PLEG): container finished" podID="2d59940d-9509-4afb-92bd-0a37c249aaa5" containerID="10e8ef5c75831f57eb2458729befded000db3b42db58c3f4565f26a8db237968" exitCode=0
Sep 08 20:27:29.479411 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:29.478942    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n" event={"ID":"2d59940d-9509-4afb-92bd-0a37c249aaa5","Type":"ContainerDied","Data":"10e8ef5c75831f57eb2458729befded000db3b42db58c3f4565f26a8db237968"}
Sep 08 20:27:29.479411 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:29.479282    2580 scope.go:117] "RemoveContainer" containerID="10e8ef5c75831f57eb2458729befded000db3b42db58c3f4565f26a8db237968"
Sep 08 20:27:30.487065 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:30.487016    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-nnl6n" event={"ID":"2d59940d-9509-4afb-92bd-0a37c249aaa5","Type":"ContainerStarted","Data":"12c28126a4663ce30409978fed82b33ec89143f7b3514b143bab4656fe77ea22"}
Sep 08 20:27:34.911749 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:34.911706    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:34.912254 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:34.911778    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:36.759279 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:36.759246    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:36.808745 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:36.808694    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs\") pod \"network-metrics-daemon-vlx4d\" (UID: \"642122c1-c2a9-4fe3-aade-12ac05cecf17\") " pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:27:36.811065 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:36.811035    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/642122c1-c2a9-4fe3-aade-12ac05cecf17-metrics-certs\") pod \"network-metrics-daemon-vlx4d\" (UID: \"642122c1-c2a9-4fe3-aade-12ac05cecf17\") " pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:27:37.041220 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:37.041140    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kbhtv\""
Sep 08 20:27:37.049802 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:37.049776    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vlx4d"
Sep 08 20:27:37.316937 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:37.316911    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vlx4d"]
Sep 08 20:27:37.319331 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:27:37.319303    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod642122c1_c2a9_4fe3_aade_12ac05cecf17.slice/crio-088eb4bbe8b1190b0f96018d81b8346c8c20b544b4c2ca63a87c6e58c8c03273 WatchSource:0}: Error finding container 088eb4bbe8b1190b0f96018d81b8346c8c20b544b4c2ca63a87c6e58c8c03273: Status 404 returned error can't find the container with id 088eb4bbe8b1190b0f96018d81b8346c8c20b544b4c2ca63a87c6e58c8c03273
Sep 08 20:27:37.514678 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:37.514640    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vlx4d" event={"ID":"642122c1-c2a9-4fe3-aade-12ac05cecf17","Type":"ContainerStarted","Data":"088eb4bbe8b1190b0f96018d81b8346c8c20b544b4c2ca63a87c6e58c8c03273"}
Sep 08 20:27:38.520071 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:38.520038    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vlx4d" event={"ID":"642122c1-c2a9-4fe3-aade-12ac05cecf17","Type":"ContainerStarted","Data":"b78d90c54f44e77db4388030016a268c00dbc5c4d13a90f63274e7d72a8e39cb"}
Sep 08 20:27:38.520481 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:38.520093    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vlx4d" event={"ID":"642122c1-c2a9-4fe3-aade-12ac05cecf17","Type":"ContainerStarted","Data":"ebd558ac43c95c946f101533c33e4585a14404ab2d0c779f9bde67054b64a7d3"}
Sep 08 20:27:38.535434 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:38.535387    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vlx4d" podStartSLOduration=130.559968002 podStartE2EDuration="2m11.535372155s" totalImagesPullingTime="975.404153ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:27:38.534804353 +0000 UTC m=+132.082053834" watchObservedRunningTime="2026-09-08 20:27:38.535372155 +0000 UTC m=+132.082621637"
Sep 08 20:27:38.616255 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:38.616135    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:38.679378 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:38.679332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:39.525439 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:39.525406    2580 generic.go:356] "Generic (PLEG): container finished" podID="d374121e-fdb4-4528-bee6-ffd573438cf1" containerID="82523636fcc08bfd4376032229170d9110781a61367c1ff213825dcaca801d3e" exitCode=0
Sep 08 20:27:39.525839 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:39.525475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl" event={"ID":"d374121e-fdb4-4528-bee6-ffd573438cf1","Type":"ContainerDied","Data":"82523636fcc08bfd4376032229170d9110781a61367c1ff213825dcaca801d3e"}
Sep 08 20:27:39.525906 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:39.525891    2580 scope.go:117] "RemoveContainer" containerID="82523636fcc08bfd4376032229170d9110781a61367c1ff213825dcaca801d3e"
Sep 08 20:27:40.531111 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:40.531076    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-qw8tl" event={"ID":"d374121e-fdb4-4528-bee6-ffd573438cf1","Type":"ContainerStarted","Data":"d3c013d48909d800bc44474dedd071ee919c2686223fc48d5e286e761b51e709"}
Sep 08 20:27:51.546250 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:51.546184    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f9b546bb-rqngb" podUID="5f85dbad-2508-4c7e-aaae-c496346d6086" containerName="console" containerID="cri-o://874a3b344ab42e7ce4099a3b30f92dbf82e4ff9c72e891ccb06ab30aa40e885b" gracePeriod=15
Sep 08 20:27:51.794136 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:51.794111    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:51.947094 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:51.947057    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-oauth-serving-cert\") pod \"5f85dbad-2508-4c7e-aaae-c496346d6086\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") "
Sep 08 20:27:51.947094 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:51.947101    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-trusted-ca-bundle\") pod \"5f85dbad-2508-4c7e-aaae-c496346d6086\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") "
Sep 08 20:27:51.947349 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:51.947150    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f85dbad-2508-4c7e-aaae-c496346d6086-console-serving-cert\") pod \"5f85dbad-2508-4c7e-aaae-c496346d6086\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") "
Sep 08 20:27:51.947349 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:51.947197    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-service-ca\") pod \"5f85dbad-2508-4c7e-aaae-c496346d6086\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") "
Sep 08 20:27:51.947349 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:51.947230    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f85dbad-2508-4c7e-aaae-c496346d6086-console-oauth-config\") pod \"5f85dbad-2508-4c7e-aaae-c496346d6086\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") "
Sep 08 20:27:51.947349 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:51.947256    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-console-config\") pod \"5f85dbad-2508-4c7e-aaae-c496346d6086\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") "
Sep 08 20:27:51.947349 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:51.947309    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9xzd\" (UniqueName: \"kubernetes.io/projected/5f85dbad-2508-4c7e-aaae-c496346d6086-kube-api-access-l9xzd\") pod \"5f85dbad-2508-4c7e-aaae-c496346d6086\" (UID: \"5f85dbad-2508-4c7e-aaae-c496346d6086\") "
Sep 08 20:27:51.947693 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:51.947648    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5f85dbad-2508-4c7e-aaae-c496346d6086" (UID: "5f85dbad-2508-4c7e-aaae-c496346d6086"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:27:51.947693 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:51.947653    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-service-ca" (OuterVolumeSpecName: "service-ca") pod "5f85dbad-2508-4c7e-aaae-c496346d6086" (UID: "5f85dbad-2508-4c7e-aaae-c496346d6086"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:27:51.947693 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:51.947688    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-console-config" (OuterVolumeSpecName: "console-config") pod "5f85dbad-2508-4c7e-aaae-c496346d6086" (UID: "5f85dbad-2508-4c7e-aaae-c496346d6086"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:27:51.947876 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:51.947663    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5f85dbad-2508-4c7e-aaae-c496346d6086" (UID: "5f85dbad-2508-4c7e-aaae-c496346d6086"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:27:51.949593 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:51.949569    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f85dbad-2508-4c7e-aaae-c496346d6086-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5f85dbad-2508-4c7e-aaae-c496346d6086" (UID: "5f85dbad-2508-4c7e-aaae-c496346d6086"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:27:51.949714 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:51.949606    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f85dbad-2508-4c7e-aaae-c496346d6086-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5f85dbad-2508-4c7e-aaae-c496346d6086" (UID: "5f85dbad-2508-4c7e-aaae-c496346d6086"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:27:51.949714 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:51.949618    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f85dbad-2508-4c7e-aaae-c496346d6086-kube-api-access-l9xzd" (OuterVolumeSpecName: "kube-api-access-l9xzd") pod "5f85dbad-2508-4c7e-aaae-c496346d6086" (UID: "5f85dbad-2508-4c7e-aaae-c496346d6086"). InnerVolumeSpecName "kube-api-access-l9xzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:27:52.048331 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.048290    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9xzd\" (UniqueName: \"kubernetes.io/projected/5f85dbad-2508-4c7e-aaae-c496346d6086-kube-api-access-l9xzd\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:27:52.048331 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.048322    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-oauth-serving-cert\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:27:52.048331 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.048331    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-trusted-ca-bundle\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:27:52.048331 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.048340    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f85dbad-2508-4c7e-aaae-c496346d6086-console-serving-cert\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:27:52.048599 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.048348    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-service-ca\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:27:52.048599 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.048357    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f85dbad-2508-4c7e-aaae-c496346d6086-console-oauth-config\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:27:52.048599 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.048365    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f85dbad-2508-4c7e-aaae-c496346d6086-console-config\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:27:52.581125 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.581082    2580 generic.go:356] "Generic (PLEG): container finished" podID="5f85dbad-2508-4c7e-aaae-c496346d6086" containerID="874a3b344ab42e7ce4099a3b30f92dbf82e4ff9c72e891ccb06ab30aa40e885b" exitCode=0
Sep 08 20:27:52.581125 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.581128    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f9b546bb-rqngb"
Sep 08 20:27:52.581550 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.581171    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9b546bb-rqngb" event={"ID":"5f85dbad-2508-4c7e-aaae-c496346d6086","Type":"ContainerDied","Data":"874a3b344ab42e7ce4099a3b30f92dbf82e4ff9c72e891ccb06ab30aa40e885b"}
Sep 08 20:27:52.581550 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.581223    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f9b546bb-rqngb" event={"ID":"5f85dbad-2508-4c7e-aaae-c496346d6086","Type":"ContainerDied","Data":"1469ddeb4f7ae3b133b8b63d3e0a42aa484cff0a0b1afba9749e8e94f7704dcb"}
Sep 08 20:27:52.581550 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.581244    2580 scope.go:117] "RemoveContainer" containerID="874a3b344ab42e7ce4099a3b30f92dbf82e4ff9c72e891ccb06ab30aa40e885b"
Sep 08 20:27:52.590697 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.590679    2580 scope.go:117] "RemoveContainer" containerID="874a3b344ab42e7ce4099a3b30f92dbf82e4ff9c72e891ccb06ab30aa40e885b"
Sep 08 20:27:52.590995 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:27:52.590970    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"874a3b344ab42e7ce4099a3b30f92dbf82e4ff9c72e891ccb06ab30aa40e885b\": container with ID starting with 874a3b344ab42e7ce4099a3b30f92dbf82e4ff9c72e891ccb06ab30aa40e885b not found: ID does not exist" containerID="874a3b344ab42e7ce4099a3b30f92dbf82e4ff9c72e891ccb06ab30aa40e885b"
Sep 08 20:27:52.591037 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.591005    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874a3b344ab42e7ce4099a3b30f92dbf82e4ff9c72e891ccb06ab30aa40e885b"} err="failed to get container status \"874a3b344ab42e7ce4099a3b30f92dbf82e4ff9c72e891ccb06ab30aa40e885b\": rpc error: code = NotFound desc = could not find container \"874a3b344ab42e7ce4099a3b30f92dbf82e4ff9c72e891ccb06ab30aa40e885b\": container with ID starting with 874a3b344ab42e7ce4099a3b30f92dbf82e4ff9c72e891ccb06ab30aa40e885b not found: ID does not exist"
Sep 08 20:27:52.601893 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.601865    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f9b546bb-rqngb"]
Sep 08 20:27:52.606648 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.606624    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f9b546bb-rqngb"]
Sep 08 20:27:52.922350 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:52.922316    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f85dbad-2508-4c7e-aaae-c496346d6086" path="/var/lib/kubelet/pods/5f85dbad-2508-4c7e-aaae-c496346d6086/volumes"
Sep 08 20:27:54.917031 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:54.916996    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:54.922167 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:54.922144    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7b9ff5f76f-f5nhx"
Sep 08 20:27:56.939206 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:56.939171    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:27:57.083341 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:27:57.083308    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:28:06.267520 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:06.267480    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:28:29.318596 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:29.318553    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:28:29.519267 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:29.519226    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:28:38.896842 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:38.896788    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:28:38.995678 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:38.995644    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:28:39.235384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:39.235300    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:28:39.289283 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:39.289237    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:28:44.547601 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:44.547561    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:28:44.602427 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:44.602393    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:28:49.189994 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.189946    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-dfdb9b4f8-pbg4f"]
Sep 08 20:28:49.190367 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.190251    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f85dbad-2508-4c7e-aaae-c496346d6086" containerName="console"
Sep 08 20:28:49.190367 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.190261    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f85dbad-2508-4c7e-aaae-c496346d6086" containerName="console"
Sep 08 20:28:49.190367 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.190317    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f85dbad-2508-4c7e-aaae-c496346d6086" containerName="console"
Sep 08 20:28:49.191941 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.191925    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.207145 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.207119    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dfdb9b4f8-pbg4f"]
Sep 08 20:28:49.235744 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.235709    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpbrv\" (UniqueName: \"kubernetes.io/projected/cb075287-9192-42ff-9331-28ac0fa29065-kube-api-access-wpbrv\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.235744 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.235745    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-trusted-ca-bundle\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.235939 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.235778    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb075287-9192-42ff-9331-28ac0fa29065-console-serving-cert\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.235939 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.235816    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-console-config\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.235939 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.235854    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-oauth-serving-cert\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.235939 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.235877    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-service-ca\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.235939 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.235917    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb075287-9192-42ff-9331-28ac0fa29065-console-oauth-config\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.336805 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.336762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpbrv\" (UniqueName: \"kubernetes.io/projected/cb075287-9192-42ff-9331-28ac0fa29065-kube-api-access-wpbrv\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.336805 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.336803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-trusted-ca-bundle\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.337074 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.336847    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb075287-9192-42ff-9331-28ac0fa29065-console-serving-cert\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.337074 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.336862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-console-config\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.337074 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.336880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-oauth-serving-cert\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.337074 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.336901    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-service-ca\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.337074 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.336978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb075287-9192-42ff-9331-28ac0fa29065-console-oauth-config\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.337796 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.337767    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-console-config\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.337906 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.337832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-oauth-serving-cert\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.337906 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.337846    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-service-ca\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.337906 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.337872    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-trusted-ca-bundle\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.339409 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.339389    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb075287-9192-42ff-9331-28ac0fa29065-console-oauth-config\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.339512 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.339495    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb075287-9192-42ff-9331-28ac0fa29065-console-serving-cert\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.346902 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.346880    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpbrv\" (UniqueName: \"kubernetes.io/projected/cb075287-9192-42ff-9331-28ac0fa29065-kube-api-access-wpbrv\") pod \"console-dfdb9b4f8-pbg4f\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") " pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.500902 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.500811    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:49.785128 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.785104    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dfdb9b4f8-pbg4f"]
Sep 08 20:28:49.787419 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:28:49.787391    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb075287_9192_42ff_9331_28ac0fa29065.slice/crio-55d9ffef29bba4ac10855b9f448172186cc8af1764b54a161332f4042008f0ee WatchSource:0}: Error finding container 55d9ffef29bba4ac10855b9f448172186cc8af1764b54a161332f4042008f0ee: Status 404 returned error can't find the container with id 55d9ffef29bba4ac10855b9f448172186cc8af1764b54a161332f4042008f0ee
Sep 08 20:28:49.821825 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:49.821797    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dfdb9b4f8-pbg4f" event={"ID":"cb075287-9192-42ff-9331-28ac0fa29065","Type":"ContainerStarted","Data":"55d9ffef29bba4ac10855b9f448172186cc8af1764b54a161332f4042008f0ee"}
Sep 08 20:28:50.827379 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:50.827340    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dfdb9b4f8-pbg4f" event={"ID":"cb075287-9192-42ff-9331-28ac0fa29065","Type":"ContainerStarted","Data":"c37a5128168c28878b4db0d1faa7668db0edb19d8cb27a979c61b97cece5ff36"}
Sep 08 20:28:50.850375 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:50.850321    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-dfdb9b4f8-pbg4f" podStartSLOduration=1.850303772 podStartE2EDuration="1.850303772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:28:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:28:50.849168317 +0000 UTC m=+204.396417800" watchObservedRunningTime="2026-09-08 20:28:50.850303772 +0000 UTC m=+204.397553252"
Sep 08 20:28:56.469747 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:56.469711    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:28:56.530306 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:56.530274    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:28:59.501647 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:59.501605    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:59.502080 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:59.501678    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:59.506531 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:59.506506    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:59.864940 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:59.864855    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:28:59.913381 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:28:59.913345    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-864f669fc-wdh6q"]
Sep 08 20:29:00.026686 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:00.026648    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:29:00.086196 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:00.086163    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:29:00.154354 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:00.154320    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:29:00.770922 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:00.770854    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:29:00.871272 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:00.871237    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:29:11.940116 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:11.940059    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:29:12.039129 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:12.039094    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:29:24.933328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:24.933267    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-864f669fc-wdh6q" podUID="1298d0be-1d59-45f7-b4b0-c02254abc260" containerName="console" containerID="cri-o://d0e89d295137cebb56131598e0e5d405fa8be310227825e9f665beeb68fbd656" gracePeriod=15
Sep 08 20:29:25.171768 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.171743    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:29:25.244430 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.244341    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1298d0be-1d59-45f7-b4b0-c02254abc260-console-oauth-config\") pod \"1298d0be-1d59-45f7-b4b0-c02254abc260\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") "
Sep 08 20:29:25.244574 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.244434    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8nbf\" (UniqueName: \"kubernetes.io/projected/1298d0be-1d59-45f7-b4b0-c02254abc260-kube-api-access-q8nbf\") pod \"1298d0be-1d59-45f7-b4b0-c02254abc260\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") "
Sep 08 20:29:25.244574 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.244486    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-oauth-serving-cert\") pod \"1298d0be-1d59-45f7-b4b0-c02254abc260\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") "
Sep 08 20:29:25.244574 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.244526    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-console-config\") pod \"1298d0be-1d59-45f7-b4b0-c02254abc260\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") "
Sep 08 20:29:25.244729 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.244573    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-service-ca\") pod \"1298d0be-1d59-45f7-b4b0-c02254abc260\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") "
Sep 08 20:29:25.244729 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.244613    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-trusted-ca-bundle\") pod \"1298d0be-1d59-45f7-b4b0-c02254abc260\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") "
Sep 08 20:29:25.244729 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.244645    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1298d0be-1d59-45f7-b4b0-c02254abc260-console-serving-cert\") pod \"1298d0be-1d59-45f7-b4b0-c02254abc260\" (UID: \"1298d0be-1d59-45f7-b4b0-c02254abc260\") "
Sep 08 20:29:25.245053 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.245020    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1298d0be-1d59-45f7-b4b0-c02254abc260" (UID: "1298d0be-1d59-45f7-b4b0-c02254abc260"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:29:25.245053 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.245034    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-console-config" (OuterVolumeSpecName: "console-config") pod "1298d0be-1d59-45f7-b4b0-c02254abc260" (UID: "1298d0be-1d59-45f7-b4b0-c02254abc260"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:29:25.245185 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.245122    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1298d0be-1d59-45f7-b4b0-c02254abc260" (UID: "1298d0be-1d59-45f7-b4b0-c02254abc260"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:29:25.245185 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.245130    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-service-ca" (OuterVolumeSpecName: "service-ca") pod "1298d0be-1d59-45f7-b4b0-c02254abc260" (UID: "1298d0be-1d59-45f7-b4b0-c02254abc260"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:29:25.246563 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.246540    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1298d0be-1d59-45f7-b4b0-c02254abc260-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1298d0be-1d59-45f7-b4b0-c02254abc260" (UID: "1298d0be-1d59-45f7-b4b0-c02254abc260"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:29:25.246654 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.246598    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1298d0be-1d59-45f7-b4b0-c02254abc260-kube-api-access-q8nbf" (OuterVolumeSpecName: "kube-api-access-q8nbf") pod "1298d0be-1d59-45f7-b4b0-c02254abc260" (UID: "1298d0be-1d59-45f7-b4b0-c02254abc260"). InnerVolumeSpecName "kube-api-access-q8nbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:29:25.246736 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.246714    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1298d0be-1d59-45f7-b4b0-c02254abc260-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1298d0be-1d59-45f7-b4b0-c02254abc260" (UID: "1298d0be-1d59-45f7-b4b0-c02254abc260"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:29:25.345507 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.345465    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-console-config\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:29:25.345507 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.345499    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-service-ca\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:29:25.345507 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.345508    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-trusted-ca-bundle\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:29:25.345507 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.345517    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1298d0be-1d59-45f7-b4b0-c02254abc260-console-serving-cert\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:29:25.345765 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.345525    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1298d0be-1d59-45f7-b4b0-c02254abc260-console-oauth-config\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:29:25.345765 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.345534    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8nbf\" (UniqueName: \"kubernetes.io/projected/1298d0be-1d59-45f7-b4b0-c02254abc260-kube-api-access-q8nbf\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:29:25.345765 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.345543    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1298d0be-1d59-45f7-b4b0-c02254abc260-oauth-serving-cert\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:29:25.960690 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.960656    2580 generic.go:356] "Generic (PLEG): container finished" podID="1298d0be-1d59-45f7-b4b0-c02254abc260" containerID="d0e89d295137cebb56131598e0e5d405fa8be310227825e9f665beeb68fbd656" exitCode=0
Sep 08 20:29:25.961095 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.960702    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864f669fc-wdh6q"
Sep 08 20:29:25.961095 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.960703    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864f669fc-wdh6q" event={"ID":"1298d0be-1d59-45f7-b4b0-c02254abc260","Type":"ContainerDied","Data":"d0e89d295137cebb56131598e0e5d405fa8be310227825e9f665beeb68fbd656"}
Sep 08 20:29:25.961095 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.960820    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864f669fc-wdh6q" event={"ID":"1298d0be-1d59-45f7-b4b0-c02254abc260","Type":"ContainerDied","Data":"b26f04d456583387a034e2f08968c0ed3facb850f152cbcd5a6007c0a629bca1"}
Sep 08 20:29:25.961095 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.960846    2580 scope.go:117] "RemoveContainer" containerID="d0e89d295137cebb56131598e0e5d405fa8be310227825e9f665beeb68fbd656"
Sep 08 20:29:25.969418 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.969403    2580 scope.go:117] "RemoveContainer" containerID="d0e89d295137cebb56131598e0e5d405fa8be310227825e9f665beeb68fbd656"
Sep 08 20:29:25.969690 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:29:25.969671    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0e89d295137cebb56131598e0e5d405fa8be310227825e9f665beeb68fbd656\": container with ID starting with d0e89d295137cebb56131598e0e5d405fa8be310227825e9f665beeb68fbd656 not found: ID does not exist" containerID="d0e89d295137cebb56131598e0e5d405fa8be310227825e9f665beeb68fbd656"
Sep 08 20:29:25.969742 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.969695    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0e89d295137cebb56131598e0e5d405fa8be310227825e9f665beeb68fbd656"} err="failed to get container status \"d0e89d295137cebb56131598e0e5d405fa8be310227825e9f665beeb68fbd656\": rpc error: code = NotFound desc = could not find container \"d0e89d295137cebb56131598e0e5d405fa8be310227825e9f665beeb68fbd656\": container with ID starting with d0e89d295137cebb56131598e0e5d405fa8be310227825e9f665beeb68fbd656 not found: ID does not exist"
Sep 08 20:29:25.980721 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.980692    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-864f669fc-wdh6q"]
Sep 08 20:29:25.984892 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:25.984866    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-864f669fc-wdh6q"]
Sep 08 20:29:26.922653 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:26.922623    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1298d0be-1d59-45f7-b4b0-c02254abc260" path="/var/lib/kubelet/pods/1298d0be-1d59-45f7-b4b0-c02254abc260/volumes"
Sep 08 20:29:57.196919 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:57.196884    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:29:57.301631 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:29:57.301595    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:18.939168 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:18.939134    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:19.893340 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:19.893306    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:22.539332 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:22.539291    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:22.644005 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:22.643950    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:25.073815 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:25.073775    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:25.185595 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:25.185542    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:25.280735 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:25.280684    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:26.839575 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:26.839538    2580 kubelet.go:1648] "Image garbage collection succeeded"
Sep 08 20:30:27.197161 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:27.197124    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:27.429210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:27.429176    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:29.725516 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:29.725469    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:29.818813 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:29.818774    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:34.140722 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:34.140669    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:34.240641 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:34.240609    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:35.602842 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.602799    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:35.682108 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.682072    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zb2rm"]
Sep 08 20:30:35.682412 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.682400    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1298d0be-1d59-45f7-b4b0-c02254abc260" containerName="console"
Sep 08 20:30:35.682455 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.682414    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1298d0be-1d59-45f7-b4b0-c02254abc260" containerName="console"
Sep 08 20:30:35.682495 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.682458    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1298d0be-1d59-45f7-b4b0-c02254abc260" containerName="console"
Sep 08 20:30:35.685303 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.685285    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-zb2rm"
Sep 08 20:30:35.687878 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.687842    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 08 20:30:35.688547 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.688531    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 08 20:30:35.688665 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.688561    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-kn94t\""
Sep 08 20:30:35.697272 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.697246    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zb2rm"]
Sep 08 20:30:35.719688 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.719647    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f59d6b5f-799c-4baa-9477-91de77c4b267-bound-sa-token\") pod \"cert-manager-55b4d954d5-zb2rm\" (UID: \"f59d6b5f-799c-4baa-9477-91de77c4b267\") " pod="cert-manager/cert-manager-55b4d954d5-zb2rm"
Sep 08 20:30:35.719847 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.719702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vplz\" (UniqueName: \"kubernetes.io/projected/f59d6b5f-799c-4baa-9477-91de77c4b267-kube-api-access-4vplz\") pod \"cert-manager-55b4d954d5-zb2rm\" (UID: \"f59d6b5f-799c-4baa-9477-91de77c4b267\") " pod="cert-manager/cert-manager-55b4d954d5-zb2rm"
Sep 08 20:30:35.738851 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.738821    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:35.821015 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.820954    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f59d6b5f-799c-4baa-9477-91de77c4b267-bound-sa-token\") pod \"cert-manager-55b4d954d5-zb2rm\" (UID: \"f59d6b5f-799c-4baa-9477-91de77c4b267\") " pod="cert-manager/cert-manager-55b4d954d5-zb2rm"
Sep 08 20:30:35.821167 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.821033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vplz\" (UniqueName: \"kubernetes.io/projected/f59d6b5f-799c-4baa-9477-91de77c4b267-kube-api-access-4vplz\") pod \"cert-manager-55b4d954d5-zb2rm\" (UID: \"f59d6b5f-799c-4baa-9477-91de77c4b267\") " pod="cert-manager/cert-manager-55b4d954d5-zb2rm"
Sep 08 20:30:35.830251 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.830223    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vplz\" (UniqueName: \"kubernetes.io/projected/f59d6b5f-799c-4baa-9477-91de77c4b267-kube-api-access-4vplz\") pod \"cert-manager-55b4d954d5-zb2rm\" (UID: \"f59d6b5f-799c-4baa-9477-91de77c4b267\") " pod="cert-manager/cert-manager-55b4d954d5-zb2rm"
Sep 08 20:30:35.831068 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.831041    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:35.832111 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:35.832091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f59d6b5f-799c-4baa-9477-91de77c4b267-bound-sa-token\") pod \"cert-manager-55b4d954d5-zb2rm\" (UID: \"f59d6b5f-799c-4baa-9477-91de77c4b267\") " pod="cert-manager/cert-manager-55b4d954d5-zb2rm"
Sep 08 20:30:36.004136 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:36.004099    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-zb2rm"
Sep 08 20:30:36.273114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:36.273032    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zb2rm"]
Sep 08 20:30:36.275976 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:30:36.275935    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf59d6b5f_799c_4baa_9477_91de77c4b267.slice/crio-63f364cefbf6e4ed32bcf36b57d3a1a1f8b4f3ec5b3218f22311f17b87dc6ad7 WatchSource:0}: Error finding container 63f364cefbf6e4ed32bcf36b57d3a1a1f8b4f3ec5b3218f22311f17b87dc6ad7: Status 404 returned error can't find the container with id 63f364cefbf6e4ed32bcf36b57d3a1a1f8b4f3ec5b3218f22311f17b87dc6ad7
Sep 08 20:30:36.277871 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:36.277852    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 20:30:37.226759 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:37.226686    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-zb2rm" event={"ID":"f59d6b5f-799c-4baa-9477-91de77c4b267","Type":"ContainerStarted","Data":"63f364cefbf6e4ed32bcf36b57d3a1a1f8b4f3ec5b3218f22311f17b87dc6ad7"}
Sep 08 20:30:37.756373 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:37.756307    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2b7xh"]
Sep 08 20:30:37.759776 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:37.759753    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2b7xh"
Sep 08 20:30:37.762141 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:37.761775    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 08 20:30:37.762141 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:37.761897    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-gcvbr\""
Sep 08 20:30:37.762594 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:37.762446    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 08 20:30:37.770409 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:37.770387    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2b7xh"]
Sep 08 20:30:37.835984 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:37.835936    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kws6\" (UniqueName: \"kubernetes.io/projected/7a613109-4ccd-48a1-800a-18ff438653da-kube-api-access-5kws6\") pod \"openshift-lws-operator-fd8ccff4c-2b7xh\" (UID: \"7a613109-4ccd-48a1-800a-18ff438653da\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2b7xh"
Sep 08 20:30:37.836168 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:37.836028    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a613109-4ccd-48a1-800a-18ff438653da-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2b7xh\" (UID: \"7a613109-4ccd-48a1-800a-18ff438653da\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2b7xh"
Sep 08 20:30:37.936727 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:37.936681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kws6\" (UniqueName: \"kubernetes.io/projected/7a613109-4ccd-48a1-800a-18ff438653da-kube-api-access-5kws6\") pod \"openshift-lws-operator-fd8ccff4c-2b7xh\" (UID: \"7a613109-4ccd-48a1-800a-18ff438653da\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2b7xh"
Sep 08 20:30:37.936909 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:37.936748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a613109-4ccd-48a1-800a-18ff438653da-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2b7xh\" (UID: \"7a613109-4ccd-48a1-800a-18ff438653da\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2b7xh"
Sep 08 20:30:37.937272 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:37.937214    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a613109-4ccd-48a1-800a-18ff438653da-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2b7xh\" (UID: \"7a613109-4ccd-48a1-800a-18ff438653da\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2b7xh"
Sep 08 20:30:37.945148 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:37.945116    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kws6\" (UniqueName: \"kubernetes.io/projected/7a613109-4ccd-48a1-800a-18ff438653da-kube-api-access-5kws6\") pod \"openshift-lws-operator-fd8ccff4c-2b7xh\" (UID: \"7a613109-4ccd-48a1-800a-18ff438653da\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2b7xh"
Sep 08 20:30:38.072663 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:38.072585    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2b7xh"
Sep 08 20:30:39.764598 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:39.764518    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2b7xh"]
Sep 08 20:30:39.767666 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:30:39.767628    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a613109_4ccd_48a1_800a_18ff438653da.slice/crio-840c99fc65ab8bbc521a2f314a51d4e546113d6d02ed044c228bbe061fcfb13d WatchSource:0}: Error finding container 840c99fc65ab8bbc521a2f314a51d4e546113d6d02ed044c228bbe061fcfb13d: Status 404 returned error can't find the container with id 840c99fc65ab8bbc521a2f314a51d4e546113d6d02ed044c228bbe061fcfb13d
Sep 08 20:30:40.243309 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:40.243248    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-zb2rm" event={"ID":"f59d6b5f-799c-4baa-9477-91de77c4b267","Type":"ContainerStarted","Data":"ebaa3cab97361ac6b5d75da03e4f43bbb8ca6d09960be9ae7d3708f2ab1b682d"}
Sep 08 20:30:40.245015 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:40.244984    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2b7xh" event={"ID":"7a613109-4ccd-48a1-800a-18ff438653da","Type":"ContainerStarted","Data":"840c99fc65ab8bbc521a2f314a51d4e546113d6d02ed044c228bbe061fcfb13d"}
Sep 08 20:30:40.260509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:40.260436    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-zb2rm" podStartSLOduration=1.9655519529999999 podStartE2EDuration="5.260415248s" totalImagesPullingTime="3.294863295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:30:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:30:40.259308069 +0000 UTC m=+313.806557553" watchObservedRunningTime="2026-09-08 20:30:40.260415248 +0000 UTC m=+313.807664733"
Sep 08 20:30:42.255841 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:42.255796    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2b7xh" event={"ID":"7a613109-4ccd-48a1-800a-18ff438653da","Type":"ContainerStarted","Data":"85c7bd51290f36489c9321ddba59ca203d6705a4d320dfa74844fd0458ffe4dd"}
Sep 08 20:30:42.271742 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:42.271622    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2b7xh" podStartSLOduration=3.023540925 podStartE2EDuration="5.271605615s" totalImagesPullingTime="2.24806469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:30:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:30:42.270421191 +0000 UTC m=+315.817670685" watchObservedRunningTime="2026-09-08 20:30:42.271605615 +0000 UTC m=+315.818855097"
Sep 08 20:30:43.026044 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:43.026005    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:43.125924 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:43.125888    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:45.373972 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:45.373926    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:54.700547 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:54.700509    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:30:56.310078 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:30:56.310045    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:00.941556 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:00.941521    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:01.031995 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:01.031934    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:01.543923 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:01.543873    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:01.769976 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:01.769906    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:01.870023 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:01.869934    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:01.903138 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:01.903100    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"]
Sep 08 20:31:01.906397 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:01.906372    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:01.908714 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:01.908689    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 08 20:31:01.908850 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:01.908771    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 08 20:31:01.908850 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:01.908836    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-sv8zf\""
Sep 08 20:31:01.909327 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:01.909308    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 08 20:31:01.910505 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:01.910483    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 08 20:31:01.938290 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:01.938260    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"]
Sep 08 20:31:02.039725 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.039686    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b8d1349-f513-443d-91a4-fa50453cc156-tmp\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-gslsc\" (UID: \"2b8d1349-f513-443d-91a4-fa50453cc156\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:02.040139 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.039726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8kht\" (UniqueName: \"kubernetes.io/projected/2b8d1349-f513-443d-91a4-fa50453cc156-kube-api-access-k8kht\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-gslsc\" (UID: \"2b8d1349-f513-443d-91a4-fa50453cc156\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:02.040139 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.039757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2b8d1349-f513-443d-91a4-fa50453cc156-manifests\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-gslsc\" (UID: \"2b8d1349-f513-443d-91a4-fa50453cc156\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:02.040139 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.039822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2b8d1349-f513-443d-91a4-fa50453cc156-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-gslsc\" (UID: \"2b8d1349-f513-443d-91a4-fa50453cc156\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:02.040139 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.039862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b8d1349-f513-443d-91a4-fa50453cc156-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-gslsc\" (UID: \"2b8d1349-f513-443d-91a4-fa50453cc156\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:02.096243 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.096211    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:02.141139 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.141101    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b8d1349-f513-443d-91a4-fa50453cc156-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-gslsc\" (UID: \"2b8d1349-f513-443d-91a4-fa50453cc156\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:02.141313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.141245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b8d1349-f513-443d-91a4-fa50453cc156-tmp\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-gslsc\" (UID: \"2b8d1349-f513-443d-91a4-fa50453cc156\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:02.141313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.141266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8kht\" (UniqueName: \"kubernetes.io/projected/2b8d1349-f513-443d-91a4-fa50453cc156-kube-api-access-k8kht\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-gslsc\" (UID: \"2b8d1349-f513-443d-91a4-fa50453cc156\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:02.141313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.141283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2b8d1349-f513-443d-91a4-fa50453cc156-manifests\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-gslsc\" (UID: \"2b8d1349-f513-443d-91a4-fa50453cc156\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:02.141313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.141308    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2b8d1349-f513-443d-91a4-fa50453cc156-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-gslsc\" (UID: \"2b8d1349-f513-443d-91a4-fa50453cc156\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:02.141691 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.141652    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2b8d1349-f513-443d-91a4-fa50453cc156-manifests\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-gslsc\" (UID: \"2b8d1349-f513-443d-91a4-fa50453cc156\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:02.143554 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.143528    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b8d1349-f513-443d-91a4-fa50453cc156-tmp\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-gslsc\" (UID: \"2b8d1349-f513-443d-91a4-fa50453cc156\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:02.143751 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.143732    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b8d1349-f513-443d-91a4-fa50453cc156-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-gslsc\" (UID: \"2b8d1349-f513-443d-91a4-fa50453cc156\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:02.143852 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.143832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2b8d1349-f513-443d-91a4-fa50453cc156-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-gslsc\" (UID: \"2b8d1349-f513-443d-91a4-fa50453cc156\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:02.148691 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.148669    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8kht\" (UniqueName: \"kubernetes.io/projected/2b8d1349-f513-443d-91a4-fa50453cc156-kube-api-access-k8kht\") pod \"opendatahub-operator-controller-manager-dbd9f8d48-gslsc\" (UID: \"2b8d1349-f513-443d-91a4-fa50453cc156\") " pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:02.179816 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.179785    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:02.216753 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.216722    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:02.282975 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.282927    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:02.380627 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.380591    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:02.478134 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.478049    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:02.496283 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.496254    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"]
Sep 08 20:31:02.500051 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:31:02.500020    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b8d1349_f513_443d_91a4_fa50453cc156.slice/crio-3aae55bdac35e2bc5870d746b01cb26dff93086afdbd2e95cfebab771c1ba1db WatchSource:0}: Error finding container 3aae55bdac35e2bc5870d746b01cb26dff93086afdbd2e95cfebab771c1ba1db: Status 404 returned error can't find the container with id 3aae55bdac35e2bc5870d746b01cb26dff93086afdbd2e95cfebab771c1ba1db
Sep 08 20:31:02.553260 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.553229    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:02.657062 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.657027    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:02.754387 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.754301    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:02.856783 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:02.856749    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:03.339085 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:03.339045    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc" event={"ID":"2b8d1349-f513-443d-91a4-fa50453cc156","Type":"ContainerStarted","Data":"3aae55bdac35e2bc5870d746b01cb26dff93086afdbd2e95cfebab771c1ba1db"}
Sep 08 20:31:05.959379 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:05.959342    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:06.358143 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:06.358070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc" event={"ID":"2b8d1349-f513-443d-91a4-fa50453cc156","Type":"ContainerStarted","Data":"517d6a50bb4c8f432c4ea4de7d881c1d5cb3027c6f8a887606c5fbdf56f913f9"}
Sep 08 20:31:07.365014 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:07.364969    2580 generic.go:356] "Generic (PLEG): container finished" podID="2b8d1349-f513-443d-91a4-fa50453cc156" containerID="517d6a50bb4c8f432c4ea4de7d881c1d5cb3027c6f8a887606c5fbdf56f913f9" exitCode=0
Sep 08 20:31:07.365413 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:07.365028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc" event={"ID":"2b8d1349-f513-443d-91a4-fa50453cc156","Type":"ContainerDied","Data":"517d6a50bb4c8f432c4ea4de7d881c1d5cb3027c6f8a887606c5fbdf56f913f9"}
Sep 08 20:31:07.653490 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:07.653448    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:07.757286 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:07.757250    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:08.371633 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:08.371542    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc" event={"ID":"2b8d1349-f513-443d-91a4-fa50453cc156","Type":"ContainerStarted","Data":"054e089b83ffb2775d705e132a0625858cf4ce44db72beb1ac1a1e6ef7d77114"}
Sep 08 20:31:08.371633 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:08.371614    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:08.391508 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:08.391451    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc" podStartSLOduration=2.644157199 podStartE2EDuration="7.391435026s" totalImagesPullingTime="3.653023397s" totalInitContainerRuntime="1.09425443s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:31:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:31:08.389632641 +0000 UTC m=+341.936882122" watchObservedRunningTime="2026-09-08 20:31:08.391435026 +0000 UTC m=+341.938684506"
Sep 08 20:31:13.482312 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:13.482276    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:13.582118 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:13.582082    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:19.378735 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:19.378656    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dbd9f8d48-gslsc"
Sep 08 20:31:21.109234 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:21.109198    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:21.242684 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:21.242645    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:24.066218 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.066170    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:24.125154 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.125117    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4"]
Sep 08 20:31:24.128197 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.128172    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4"
Sep 08 20:31:24.130346 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.130273    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 08 20:31:24.130495 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.130354    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 08 20:31:24.131035 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.131018    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-srbcb\""
Sep 08 20:31:24.131169 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.131023    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 08 20:31:24.132340 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.132325    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 08 20:31:24.138300 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.138272    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4"]
Sep 08 20:31:24.235304 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.235269    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6468cbd-3da7-4625-80d4-8c4c2ef96d89-tls-certs\") pod \"kube-auth-proxy-7bf757d869-l5cr4\" (UID: \"e6468cbd-3da7-4625-80d4-8c4c2ef96d89\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4"
Sep 08 20:31:24.235466 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.235325    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6468cbd-3da7-4625-80d4-8c4c2ef96d89-tmp\") pod \"kube-auth-proxy-7bf757d869-l5cr4\" (UID: \"e6468cbd-3da7-4625-80d4-8c4c2ef96d89\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4"
Sep 08 20:31:24.235466 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.235428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmjck\" (UniqueName: \"kubernetes.io/projected/e6468cbd-3da7-4625-80d4-8c4c2ef96d89-kube-api-access-pmjck\") pod \"kube-auth-proxy-7bf757d869-l5cr4\" (UID: \"e6468cbd-3da7-4625-80d4-8c4c2ef96d89\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4"
Sep 08 20:31:24.268563 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.268512    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:24.336888 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.336810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6468cbd-3da7-4625-80d4-8c4c2ef96d89-tls-certs\") pod \"kube-auth-proxy-7bf757d869-l5cr4\" (UID: \"e6468cbd-3da7-4625-80d4-8c4c2ef96d89\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4"
Sep 08 20:31:24.336888 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.336864    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6468cbd-3da7-4625-80d4-8c4c2ef96d89-tmp\") pod \"kube-auth-proxy-7bf757d869-l5cr4\" (UID: \"e6468cbd-3da7-4625-80d4-8c4c2ef96d89\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4"
Sep 08 20:31:24.337121 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.336910    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmjck\" (UniqueName: \"kubernetes.io/projected/e6468cbd-3da7-4625-80d4-8c4c2ef96d89-kube-api-access-pmjck\") pod \"kube-auth-proxy-7bf757d869-l5cr4\" (UID: \"e6468cbd-3da7-4625-80d4-8c4c2ef96d89\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4"
Sep 08 20:31:24.339153 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.339129    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6468cbd-3da7-4625-80d4-8c4c2ef96d89-tmp\") pod \"kube-auth-proxy-7bf757d869-l5cr4\" (UID: \"e6468cbd-3da7-4625-80d4-8c4c2ef96d89\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4"
Sep 08 20:31:24.339365 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.339346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6468cbd-3da7-4625-80d4-8c4c2ef96d89-tls-certs\") pod \"kube-auth-proxy-7bf757d869-l5cr4\" (UID: \"e6468cbd-3da7-4625-80d4-8c4c2ef96d89\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4"
Sep 08 20:31:24.344050 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.344032    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmjck\" (UniqueName: \"kubernetes.io/projected/e6468cbd-3da7-4625-80d4-8c4c2ef96d89-kube-api-access-pmjck\") pod \"kube-auth-proxy-7bf757d869-l5cr4\" (UID: \"e6468cbd-3da7-4625-80d4-8c4c2ef96d89\") " pod="openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4"
Sep 08 20:31:24.439062 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.439032    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4"
Sep 08 20:31:24.711451 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:24.711288    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4"]
Sep 08 20:31:24.714224 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:31:24.714196    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6468cbd_3da7_4625_80d4_8c4c2ef96d89.slice/crio-682dc4e315344fdffef3859b249acb2fe09b1dadd9e6e54653250eb4b9bcc239 WatchSource:0}: Error finding container 682dc4e315344fdffef3859b249acb2fe09b1dadd9e6e54653250eb4b9bcc239: Status 404 returned error can't find the container with id 682dc4e315344fdffef3859b249acb2fe09b1dadd9e6e54653250eb4b9bcc239
Sep 08 20:31:25.273117 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:25.273075    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:25.375340 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:25.375301    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:25.444736 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:25.444696    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4" event={"ID":"e6468cbd-3da7-4625-80d4-8c4c2ef96d89","Type":"ContainerStarted","Data":"682dc4e315344fdffef3859b249acb2fe09b1dadd9e6e54653250eb4b9bcc239"}
Sep 08 20:31:26.251409 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:26.251372    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:27.703014 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:27.702985    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 08 20:31:28.460750 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:28.460714    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4" event={"ID":"e6468cbd-3da7-4625-80d4-8c4c2ef96d89","Type":"ContainerStarted","Data":"96cadfd2c8f72f06c3b5e2edf9e5683dfaaf68c315bfa0ce45e8c4f8a2bd6582"}
Sep 08 20:31:28.476687 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:28.476613    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7bf757d869-l5cr4" podStartSLOduration=1.491927517 podStartE2EDuration="4.476593056s" totalImagesPullingTime="2.984665539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:31:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:31:28.476234936 +0000 UTC m=+362.023484417" watchObservedRunningTime="2026-09-08 20:31:28.476593056 +0000 UTC m=+362.023842541"
Sep 08 20:31:30.287322 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:30.287284    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:30.390071 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:30.390039    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:31.049359 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.049325    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"]
Sep 08 20:31:31.052664 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.052638    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.054797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.054771    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 08 20:31:31.054915 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.054771    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-4thwp\""
Sep 08 20:31:31.063876 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.063848    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"]
Sep 08 20:31:31.088655 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.088622    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:31.092882 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.092855    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5be66a0-cfc9-4745-a2a6-83f195bfe307-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.093023 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.092897    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5be66a0-cfc9-4745-a2a6-83f195bfe307-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.093023 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.092993    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5be66a0-cfc9-4745-a2a6-83f195bfe307-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.093023 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.093018    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5be66a0-cfc9-4745-a2a6-83f195bfe307-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.093180 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.093069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5be66a0-cfc9-4745-a2a6-83f195bfe307-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.093180 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.093102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5be66a0-cfc9-4745-a2a6-83f195bfe307-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.093180 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.093126    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tzrl\" (UniqueName: \"kubernetes.io/projected/f5be66a0-cfc9-4745-a2a6-83f195bfe307-kube-api-access-6tzrl\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.093272 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.093179    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5be66a0-cfc9-4745-a2a6-83f195bfe307-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.093272 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.093203    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5be66a0-cfc9-4745-a2a6-83f195bfe307-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.193927 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.193876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5be66a0-cfc9-4745-a2a6-83f195bfe307-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.193927 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.193928    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5be66a0-cfc9-4745-a2a6-83f195bfe307-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.194198 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.193985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5be66a0-cfc9-4745-a2a6-83f195bfe307-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.194198 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.194012    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5be66a0-cfc9-4745-a2a6-83f195bfe307-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.194198 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.194045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5be66a0-cfc9-4745-a2a6-83f195bfe307-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.194198 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.194067    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5be66a0-cfc9-4745-a2a6-83f195bfe307-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.194198 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.194117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5be66a0-cfc9-4745-a2a6-83f195bfe307-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.194198 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.194142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5be66a0-cfc9-4745-a2a6-83f195bfe307-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.194198 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.194160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tzrl\" (UniqueName: \"kubernetes.io/projected/f5be66a0-cfc9-4745-a2a6-83f195bfe307-kube-api-access-6tzrl\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.194553 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.194402    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5be66a0-cfc9-4745-a2a6-83f195bfe307-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.194553 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.194465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5be66a0-cfc9-4745-a2a6-83f195bfe307-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.194553 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.194503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5be66a0-cfc9-4745-a2a6-83f195bfe307-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.194661 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.194617    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5be66a0-cfc9-4745-a2a6-83f195bfe307-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.194977 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.194934    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5be66a0-cfc9-4745-a2a6-83f195bfe307-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.196437 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.196404    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5be66a0-cfc9-4745-a2a6-83f195bfe307-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.196608 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.196589    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5be66a0-cfc9-4745-a2a6-83f195bfe307-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.201941 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.201915    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tzrl\" (UniqueName: \"kubernetes.io/projected/f5be66a0-cfc9-4745-a2a6-83f195bfe307-kube-api-access-6tzrl\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.202086 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.202069    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5be66a0-cfc9-4745-a2a6-83f195bfe307-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7\" (UID: \"f5be66a0-cfc9-4745-a2a6-83f195bfe307\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.277492 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.277455    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:31.362949 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.362841    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:31.376569 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.376541    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:31.644428 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:31.644395    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"]
Sep 08 20:31:31.647563 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:31:31.647538    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5be66a0_cfc9_4745_a2a6_83f195bfe307.slice/crio-1201c15827725178acd6e84832524bbf5885128c2df9907c17002e341bde2dbb WatchSource:0}: Error finding container 1201c15827725178acd6e84832524bbf5885128c2df9907c17002e341bde2dbb: Status 404 returned error can't find the container with id 1201c15827725178acd6e84832524bbf5885128c2df9907c17002e341bde2dbb
Sep 08 20:31:32.479153 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:32.479120    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7" event={"ID":"f5be66a0-cfc9-4745-a2a6-83f195bfe307","Type":"ContainerStarted","Data":"1201c15827725178acd6e84832524bbf5885128c2df9907c17002e341bde2dbb"}
Sep 08 20:31:32.505634 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:32.505598    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tvxbb"]
Sep 08 20:31:32.510186 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:32.510169    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-tvxbb"
Sep 08 20:31:32.512297 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:32.512265    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 08 20:31:32.512297 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:32.512285    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-xhmtb\""
Sep 08 20:31:32.519357 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:32.519330    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tvxbb"]
Sep 08 20:31:32.524169 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:32.524147    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:32.589681 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:32.589645    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:32.605085 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:32.605059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8tq4\" (UniqueName: \"kubernetes.io/projected/30c86469-f125-4b88-9219-11e5745f58c1-kube-api-access-m8tq4\") pod \"odh-model-controller-7998c67dc4-tvxbb\" (UID: \"30c86469-f125-4b88-9219-11e5745f58c1\") " pod="opendatahub/odh-model-controller-7998c67dc4-tvxbb"
Sep 08 20:31:32.605220 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:32.605102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30c86469-f125-4b88-9219-11e5745f58c1-cert\") pod \"odh-model-controller-7998c67dc4-tvxbb\" (UID: \"30c86469-f125-4b88-9219-11e5745f58c1\") " pod="opendatahub/odh-model-controller-7998c67dc4-tvxbb"
Sep 08 20:31:32.693264 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:32.693179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:32.706372 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:32.706337    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8tq4\" (UniqueName: \"kubernetes.io/projected/30c86469-f125-4b88-9219-11e5745f58c1-kube-api-access-m8tq4\") pod \"odh-model-controller-7998c67dc4-tvxbb\" (UID: \"30c86469-f125-4b88-9219-11e5745f58c1\") " pod="opendatahub/odh-model-controller-7998c67dc4-tvxbb"
Sep 08 20:31:32.706526 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:32.706411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30c86469-f125-4b88-9219-11e5745f58c1-cert\") pod \"odh-model-controller-7998c67dc4-tvxbb\" (UID: \"30c86469-f125-4b88-9219-11e5745f58c1\") " pod="opendatahub/odh-model-controller-7998c67dc4-tvxbb"
Sep 08 20:31:32.706621 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:31:32.706563    2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 08 20:31:32.706673 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:31:32.706646    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30c86469-f125-4b88-9219-11e5745f58c1-cert podName:30c86469-f125-4b88-9219-11e5745f58c1 nodeName:}" failed. No retries permitted until 2026-09-08 20:31:33.206621935 +0000 UTC m=+366.753871399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/30c86469-f125-4b88-9219-11e5745f58c1-cert") pod "odh-model-controller-7998c67dc4-tvxbb" (UID: "30c86469-f125-4b88-9219-11e5745f58c1") : secret "odh-model-controller-webhook-cert" not found
Sep 08 20:31:32.715328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:32.715301    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8tq4\" (UniqueName: \"kubernetes.io/projected/30c86469-f125-4b88-9219-11e5745f58c1-kube-api-access-m8tq4\") pod \"odh-model-controller-7998c67dc4-tvxbb\" (UID: \"30c86469-f125-4b88-9219-11e5745f58c1\") " pod="opendatahub/odh-model-controller-7998c67dc4-tvxbb"
Sep 08 20:31:32.795000 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:32.794862    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:32.890859 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:32.890810    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:33.010630 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.010592    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-75tzr"]
Sep 08 20:31:33.017856 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.017828    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-75tzr"
Sep 08 20:31:33.020338 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.020311    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 08 20:31:33.025141 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.025083    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-75tzr"]
Sep 08 20:31:33.029829 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.029800    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:33.110291 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.110197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e235df69-f4bb-4205-8e69-bccafc3b1eec-tls-certs\") pod \"model-serving-api-54d89bd5fc-75tzr\" (UID: \"e235df69-f4bb-4205-8e69-bccafc3b1eec\") " pod="opendatahub/model-serving-api-54d89bd5fc-75tzr"
Sep 08 20:31:33.110291 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.110243    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bw7v\" (UniqueName: \"kubernetes.io/projected/e235df69-f4bb-4205-8e69-bccafc3b1eec-kube-api-access-7bw7v\") pod \"model-serving-api-54d89bd5fc-75tzr\" (UID: \"e235df69-f4bb-4205-8e69-bccafc3b1eec\") " pod="opendatahub/model-serving-api-54d89bd5fc-75tzr"
Sep 08 20:31:33.211237 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.210700    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e235df69-f4bb-4205-8e69-bccafc3b1eec-tls-certs\") pod \"model-serving-api-54d89bd5fc-75tzr\" (UID: \"e235df69-f4bb-4205-8e69-bccafc3b1eec\") " pod="opendatahub/model-serving-api-54d89bd5fc-75tzr"
Sep 08 20:31:33.211237 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.210754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bw7v\" (UniqueName: \"kubernetes.io/projected/e235df69-f4bb-4205-8e69-bccafc3b1eec-kube-api-access-7bw7v\") pod \"model-serving-api-54d89bd5fc-75tzr\" (UID: \"e235df69-f4bb-4205-8e69-bccafc3b1eec\") " pod="opendatahub/model-serving-api-54d89bd5fc-75tzr"
Sep 08 20:31:33.211237 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.210894    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30c86469-f125-4b88-9219-11e5745f58c1-cert\") pod \"odh-model-controller-7998c67dc4-tvxbb\" (UID: \"30c86469-f125-4b88-9219-11e5745f58c1\") " pod="opendatahub/odh-model-controller-7998c67dc4-tvxbb"
Sep 08 20:31:33.213655 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.213625    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30c86469-f125-4b88-9219-11e5745f58c1-cert\") pod \"odh-model-controller-7998c67dc4-tvxbb\" (UID: \"30c86469-f125-4b88-9219-11e5745f58c1\") " pod="opendatahub/odh-model-controller-7998c67dc4-tvxbb"
Sep 08 20:31:33.213788 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.213734    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e235df69-f4bb-4205-8e69-bccafc3b1eec-tls-certs\") pod \"model-serving-api-54d89bd5fc-75tzr\" (UID: \"e235df69-f4bb-4205-8e69-bccafc3b1eec\") " pod="opendatahub/model-serving-api-54d89bd5fc-75tzr"
Sep 08 20:31:33.218722 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.218696    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bw7v\" (UniqueName: \"kubernetes.io/projected/e235df69-f4bb-4205-8e69-bccafc3b1eec-kube-api-access-7bw7v\") pod \"model-serving-api-54d89bd5fc-75tzr\" (UID: \"e235df69-f4bb-4205-8e69-bccafc3b1eec\") " pod="opendatahub/model-serving-api-54d89bd5fc-75tzr"
Sep 08 20:31:33.245363 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.245332    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:33.332649 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.332609    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-75tzr"
Sep 08 20:31:33.421667 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.421632    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-tvxbb"
Sep 08 20:31:33.611443 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.611311    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-75tzr"]
Sep 08 20:31:33.613931 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:31:33.613887    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode235df69_f4bb_4205_8e69_bccafc3b1eec.slice/crio-c84129608a3c529a71943b5d91b6cc38e9e1feb382a0549b56b08c92b8884cfc WatchSource:0}: Error finding container c84129608a3c529a71943b5d91b6cc38e9e1feb382a0549b56b08c92b8884cfc: Status 404 returned error can't find the container with id c84129608a3c529a71943b5d91b6cc38e9e1feb382a0549b56b08c92b8884cfc
Sep 08 20:31:33.702373 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:33.702167    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tvxbb"]
Sep 08 20:31:33.704985 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:31:33.704946    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30c86469_f125_4b88_9219_11e5745f58c1.slice/crio-764af566b67adf1c2038fc242c909d080a60ef18ee4189dd8ddf2628135f1fc7 WatchSource:0}: Error finding container 764af566b67adf1c2038fc242c909d080a60ef18ee4189dd8ddf2628135f1fc7: Status 404 returned error can't find the container with id 764af566b67adf1c2038fc242c909d080a60ef18ee4189dd8ddf2628135f1fc7
Sep 08 20:31:34.492975 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:34.492923    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-75tzr" event={"ID":"e235df69-f4bb-4205-8e69-bccafc3b1eec","Type":"ContainerStarted","Data":"c84129608a3c529a71943b5d91b6cc38e9e1feb382a0549b56b08c92b8884cfc"}
Sep 08 20:31:34.494348 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:34.494322    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-tvxbb" event={"ID":"30c86469-f125-4b88-9219-11e5745f58c1","Type":"ContainerStarted","Data":"764af566b67adf1c2038fc242c909d080a60ef18ee4189dd8ddf2628135f1fc7"}
Sep 08 20:31:34.783637 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:34.783588    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:31:34.783950 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:34.783671    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:31:34.783950 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:34.783701    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:31:35.504012 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:35.503693    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7" event={"ID":"f5be66a0-cfc9-4745-a2a6-83f195bfe307","Type":"ContainerStarted","Data":"fabd5af7fba2b19ae1081d5dc4f5e9dfcc87792702eb38a282946694f11a875b"}
Sep 08 20:31:35.525802 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:35.525594    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7" podStartSLOduration=1.3918920350000001 podStartE2EDuration="4.525572141s" totalImagesPullingTime="3.133680106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:31:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:31:35.524947568 +0000 UTC m=+369.072197049" watchObservedRunningTime="2026-09-08 20:31:35.525572141 +0000 UTC m=+369.072821623"
Sep 08 20:31:36.363246 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:36.363202    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:36.368687 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:36.368660    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:36.509376 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:36.509323    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:36.510446 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:36.510420    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7"
Sep 08 20:31:37.515308 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:37.515260    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-75tzr" event={"ID":"e235df69-f4bb-4205-8e69-bccafc3b1eec","Type":"ContainerStarted","Data":"5d9e238c61f5333ae18f69d1a5f9d19353cc9a574ea7e227b3275a7a48fe49e8"}
Sep 08 20:31:37.515742 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:37.515337    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-75tzr"
Sep 08 20:31:37.516774 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:37.516751    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-tvxbb" event={"ID":"30c86469-f125-4b88-9219-11e5745f58c1","Type":"ContainerStarted","Data":"6ec7258bcf888e7a1b32526639213c5882f4c125c9d2613fce2ab5d5c142825c"}
Sep 08 20:31:37.517042 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:37.517023    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-tvxbb"
Sep 08 20:31:37.531782 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:37.531689    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-75tzr" podStartSLOduration=1.8709258530000001 podStartE2EDuration="5.531676819s" totalImagesPullingTime="3.660750966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:31:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:31:37.53093367 +0000 UTC m=+371.078183152" watchObservedRunningTime="2026-09-08 20:31:37.531676819 +0000 UTC m=+371.078926300"
Sep 08 20:31:37.546696 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:37.546642    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-tvxbb" podStartSLOduration=1.949740831 podStartE2EDuration="5.546626587s" totalImagesPullingTime="3.596885756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:31:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:31:37.545049397 +0000 UTC m=+371.092298880" watchObservedRunningTime="2026-09-08 20:31:37.546626587 +0000 UTC m=+371.093876112"
Sep 08 20:31:38.470456 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.470408    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"]
Sep 08 20:31:38.472697 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.472681    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"
Sep 08 20:31:38.475035 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.475007    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 08 20:31:38.475712 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.475694    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 08 20:31:38.475788 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.475732    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-42k6v\""
Sep 08 20:31:38.475788 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.475744    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 08 20:31:38.483368 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.483346    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"]
Sep 08 20:31:38.560336 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.560292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/96fe2364-3ed4-4515-838f-736edf76ee93-metrics-cert\") pod \"lws-controller-manager-57c49bb89d-snfvp\" (UID: \"96fe2364-3ed4-4515-838f-736edf76ee93\") " pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"
Sep 08 20:31:38.560806 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.560372    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5jl5\" (UniqueName: \"kubernetes.io/projected/96fe2364-3ed4-4515-838f-736edf76ee93-kube-api-access-t5jl5\") pod \"lws-controller-manager-57c49bb89d-snfvp\" (UID: \"96fe2364-3ed4-4515-838f-736edf76ee93\") " pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"
Sep 08 20:31:38.560806 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.560525    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96fe2364-3ed4-4515-838f-736edf76ee93-cert\") pod \"lws-controller-manager-57c49bb89d-snfvp\" (UID: \"96fe2364-3ed4-4515-838f-736edf76ee93\") " pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"
Sep 08 20:31:38.560910 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.560819    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/96fe2364-3ed4-4515-838f-736edf76ee93-manager-config\") pod \"lws-controller-manager-57c49bb89d-snfvp\" (UID: \"96fe2364-3ed4-4515-838f-736edf76ee93\") " pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"
Sep 08 20:31:38.661446 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.661405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/96fe2364-3ed4-4515-838f-736edf76ee93-metrics-cert\") pod \"lws-controller-manager-57c49bb89d-snfvp\" (UID: \"96fe2364-3ed4-4515-838f-736edf76ee93\") " pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"
Sep 08 20:31:38.661633 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.661455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5jl5\" (UniqueName: \"kubernetes.io/projected/96fe2364-3ed4-4515-838f-736edf76ee93-kube-api-access-t5jl5\") pod \"lws-controller-manager-57c49bb89d-snfvp\" (UID: \"96fe2364-3ed4-4515-838f-736edf76ee93\") " pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"
Sep 08 20:31:38.661633 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.661513    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96fe2364-3ed4-4515-838f-736edf76ee93-cert\") pod \"lws-controller-manager-57c49bb89d-snfvp\" (UID: \"96fe2364-3ed4-4515-838f-736edf76ee93\") " pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"
Sep 08 20:31:38.661633 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.661575    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/96fe2364-3ed4-4515-838f-736edf76ee93-manager-config\") pod \"lws-controller-manager-57c49bb89d-snfvp\" (UID: \"96fe2364-3ed4-4515-838f-736edf76ee93\") " pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"
Sep 08 20:31:38.662275 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.662248    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/96fe2364-3ed4-4515-838f-736edf76ee93-manager-config\") pod \"lws-controller-manager-57c49bb89d-snfvp\" (UID: \"96fe2364-3ed4-4515-838f-736edf76ee93\") " pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"
Sep 08 20:31:38.664041 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.664021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/96fe2364-3ed4-4515-838f-736edf76ee93-metrics-cert\") pod \"lws-controller-manager-57c49bb89d-snfvp\" (UID: \"96fe2364-3ed4-4515-838f-736edf76ee93\") " pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"
Sep 08 20:31:38.664136 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.664114    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96fe2364-3ed4-4515-838f-736edf76ee93-cert\") pod \"lws-controller-manager-57c49bb89d-snfvp\" (UID: \"96fe2364-3ed4-4515-838f-736edf76ee93\") " pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"
Sep 08 20:31:38.670491 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.670465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5jl5\" (UniqueName: \"kubernetes.io/projected/96fe2364-3ed4-4515-838f-736edf76ee93-kube-api-access-t5jl5\") pod \"lws-controller-manager-57c49bb89d-snfvp\" (UID: \"96fe2364-3ed4-4515-838f-736edf76ee93\") " pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"
Sep 08 20:31:38.782890 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:38.782797    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"
Sep 08 20:31:39.067256 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:39.067227    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"]
Sep 08 20:31:39.069375 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:31:39.069352    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96fe2364_3ed4_4515_838f_736edf76ee93.slice/crio-7705f03d46d69728698d273ca85dc3f43e0a5e13281ca3be807c1518d692e43b WatchSource:0}: Error finding container 7705f03d46d69728698d273ca85dc3f43e0a5e13281ca3be807c1518d692e43b: Status 404 returned error can't find the container with id 7705f03d46d69728698d273ca85dc3f43e0a5e13281ca3be807c1518d692e43b
Sep 08 20:31:39.527181 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:39.527137    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp" event={"ID":"96fe2364-3ed4-4515-838f-736edf76ee93","Type":"ContainerStarted","Data":"7705f03d46d69728698d273ca85dc3f43e0a5e13281ca3be807c1518d692e43b"}
Sep 08 20:31:41.537868 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:41.537830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp" event={"ID":"96fe2364-3ed4-4515-838f-736edf76ee93","Type":"ContainerStarted","Data":"80934115ab946d2798cea5fa9a7a4aa563384ffed0a556b94b9059f8b125e494"}
Sep 08 20:31:41.538359 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:41.537900    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"
Sep 08 20:31:41.558887 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:41.558835    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp" podStartSLOduration=1.740468146 podStartE2EDuration="3.558818872s" totalImagesPullingTime="1.818350726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:31:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:31:41.556783934 +0000 UTC m=+375.104033414" watchObservedRunningTime="2026-09-08 20:31:41.558818872 +0000 UTC m=+375.106068353"
Sep 08 20:31:42.440469 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:42.440439    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:42.532202 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:42.532169    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:42.607647 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:42.607607    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:43.092629 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:43.092596    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:43.798413 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:43.798382    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:43.895261 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:43.895219    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:48.524540 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:48.524502    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-tvxbb"
Sep 08 20:31:48.526398 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:48.526373    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-75tzr"
Sep 08 20:31:52.545673 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:52.545638    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57c49bb89d-snfvp"
Sep 08 20:31:53.998846 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:53.998814    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:31:54.125888 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:31:54.125832    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:04.396630 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:04.396584    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dfdb9b4f8-pbg4f"]
Sep 08 20:32:04.501617 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:04.501579    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:04.571164 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:04.571128    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:04.626107 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:04.626070    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:29.416531 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.416490    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-dfdb9b4f8-pbg4f" podUID="cb075287-9192-42ff-9331-28ac0fa29065" containerName="console" containerID="cri-o://c37a5128168c28878b4db0d1faa7668db0edb19d8cb27a979c61b97cece5ff36" gracePeriod=15
Sep 08 20:32:29.671863 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.671795    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:32:29.719732 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.719693    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-console-config\") pod \"cb075287-9192-42ff-9331-28ac0fa29065\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") "
Sep 08 20:32:29.719911 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.719748    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-trusted-ca-bundle\") pod \"cb075287-9192-42ff-9331-28ac0fa29065\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") "
Sep 08 20:32:29.719911 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.719789    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-oauth-serving-cert\") pod \"cb075287-9192-42ff-9331-28ac0fa29065\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") "
Sep 08 20:32:29.719911 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.719812    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpbrv\" (UniqueName: \"kubernetes.io/projected/cb075287-9192-42ff-9331-28ac0fa29065-kube-api-access-wpbrv\") pod \"cb075287-9192-42ff-9331-28ac0fa29065\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") "
Sep 08 20:32:29.719911 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.719838    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-service-ca\") pod \"cb075287-9192-42ff-9331-28ac0fa29065\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") "
Sep 08 20:32:29.719911 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.719869    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb075287-9192-42ff-9331-28ac0fa29065-console-oauth-config\") pod \"cb075287-9192-42ff-9331-28ac0fa29065\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") "
Sep 08 20:32:29.719911 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.719898    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb075287-9192-42ff-9331-28ac0fa29065-console-serving-cert\") pod \"cb075287-9192-42ff-9331-28ac0fa29065\" (UID: \"cb075287-9192-42ff-9331-28ac0fa29065\") "
Sep 08 20:32:29.720245 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.720216    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-console-config" (OuterVolumeSpecName: "console-config") pod "cb075287-9192-42ff-9331-28ac0fa29065" (UID: "cb075287-9192-42ff-9331-28ac0fa29065"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:32:29.720300 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.720286    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cb075287-9192-42ff-9331-28ac0fa29065" (UID: "cb075287-9192-42ff-9331-28ac0fa29065"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:32:29.720347 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.720296    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-service-ca" (OuterVolumeSpecName: "service-ca") pod "cb075287-9192-42ff-9331-28ac0fa29065" (UID: "cb075287-9192-42ff-9331-28ac0fa29065"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:32:29.720389 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.720347    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cb075287-9192-42ff-9331-28ac0fa29065" (UID: "cb075287-9192-42ff-9331-28ac0fa29065"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:32:29.722095 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.722062    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb075287-9192-42ff-9331-28ac0fa29065-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cb075287-9192-42ff-9331-28ac0fa29065" (UID: "cb075287-9192-42ff-9331-28ac0fa29065"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:32:29.722095 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.722075    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb075287-9192-42ff-9331-28ac0fa29065-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cb075287-9192-42ff-9331-28ac0fa29065" (UID: "cb075287-9192-42ff-9331-28ac0fa29065"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:32:29.722241 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.722203    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb075287-9192-42ff-9331-28ac0fa29065-kube-api-access-wpbrv" (OuterVolumeSpecName: "kube-api-access-wpbrv") pod "cb075287-9192-42ff-9331-28ac0fa29065" (UID: "cb075287-9192-42ff-9331-28ac0fa29065"). InnerVolumeSpecName "kube-api-access-wpbrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:32:29.767414 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.767381    2580 generic.go:356] "Generic (PLEG): container finished" podID="cb075287-9192-42ff-9331-28ac0fa29065" containerID="c37a5128168c28878b4db0d1faa7668db0edb19d8cb27a979c61b97cece5ff36" exitCode=0
Sep 08 20:32:29.767588 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.767423    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dfdb9b4f8-pbg4f"
Sep 08 20:32:29.767588 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.767468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dfdb9b4f8-pbg4f" event={"ID":"cb075287-9192-42ff-9331-28ac0fa29065","Type":"ContainerDied","Data":"c37a5128168c28878b4db0d1faa7668db0edb19d8cb27a979c61b97cece5ff36"}
Sep 08 20:32:29.767588 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.767522    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dfdb9b4f8-pbg4f" event={"ID":"cb075287-9192-42ff-9331-28ac0fa29065","Type":"ContainerDied","Data":"55d9ffef29bba4ac10855b9f448172186cc8af1764b54a161332f4042008f0ee"}
Sep 08 20:32:29.767588 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.767543    2580 scope.go:117] "RemoveContainer" containerID="c37a5128168c28878b4db0d1faa7668db0edb19d8cb27a979c61b97cece5ff36"
Sep 08 20:32:29.776284 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.776263    2580 scope.go:117] "RemoveContainer" containerID="c37a5128168c28878b4db0d1faa7668db0edb19d8cb27a979c61b97cece5ff36"
Sep 08 20:32:29.776568 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:32:29.776551    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c37a5128168c28878b4db0d1faa7668db0edb19d8cb27a979c61b97cece5ff36\": container with ID starting with c37a5128168c28878b4db0d1faa7668db0edb19d8cb27a979c61b97cece5ff36 not found: ID does not exist" containerID="c37a5128168c28878b4db0d1faa7668db0edb19d8cb27a979c61b97cece5ff36"
Sep 08 20:32:29.776618 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.776575    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c37a5128168c28878b4db0d1faa7668db0edb19d8cb27a979c61b97cece5ff36"} err="failed to get container status \"c37a5128168c28878b4db0d1faa7668db0edb19d8cb27a979c61b97cece5ff36\": rpc error: code = NotFound desc = could not find container \"c37a5128168c28878b4db0d1faa7668db0edb19d8cb27a979c61b97cece5ff36\": container with ID starting with c37a5128168c28878b4db0d1faa7668db0edb19d8cb27a979c61b97cece5ff36 not found: ID does not exist"
Sep 08 20:32:29.787396 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.787362    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dfdb9b4f8-pbg4f"]
Sep 08 20:32:29.791316 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.791289    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-dfdb9b4f8-pbg4f"]
Sep 08 20:32:29.821122 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.821082    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-console-config\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:32:29.821122 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.821114    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-trusted-ca-bundle\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:32:29.821122 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.821124    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-oauth-serving-cert\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:32:29.821357 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.821133    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpbrv\" (UniqueName: \"kubernetes.io/projected/cb075287-9192-42ff-9331-28ac0fa29065-kube-api-access-wpbrv\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:32:29.821357 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.821142    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb075287-9192-42ff-9331-28ac0fa29065-service-ca\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:32:29.821357 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.821150    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb075287-9192-42ff-9331-28ac0fa29065-console-oauth-config\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:32:29.821357 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:29.821158    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb075287-9192-42ff-9331-28ac0fa29065-console-serving-cert\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:32:30.923587 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:30.923542    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb075287-9192-42ff-9331-28ac0fa29065" path="/var/lib/kubelet/pods/cb075287-9192-42ff-9331-28ac0fa29065/volumes"
Sep 08 20:32:33.417059 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:33.417025    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:33.511541 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:33.511509    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:47.359821 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:47.359773    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:48.091635 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:48.091604    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:48.905110 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:48.905070    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:51.695932 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:51.695897    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:52.016995 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:52.016901    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:52.125900 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:52.125851    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:52.724845 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:52.724809    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:55.106330 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:55.106296    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:55.702485 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:55.702448    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:59.385272 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:59.385236    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:59.613460 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:59.613426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:32:59.727458 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:32:59.727374    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:05.103015 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:05.102953    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:05.208924 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:05.208887    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:05.710349 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:05.710316    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:05.789941 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:05.789908    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:06.308268 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:06.308231    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:06.367982 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:06.367922    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:09.489232 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:09.489190    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:13.959770 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:13.959738    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:14.139032 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:14.138997    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:15.134034 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:15.134001    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:16.152512 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:16.152481    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:19.498953 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:19.498901    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:19.598970 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:19.598934    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:19.945730 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:19.945692    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:20.047170 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:20.047135    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:23.487310 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:23.487271    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:23.580891 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:23.580848    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:27.695482 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:27.695448    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:27.795802 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:27.795772    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:27.900355 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:27.900318    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:27.959124 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:27.959026    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:28.103383 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:28.103346    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:28.203204 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:28.203173    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:33.662496 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:33.662460    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:33.725152 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:33.725113    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:40.212743 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:40.212708    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:40.325168 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:40.325135    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:40.756613 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:40.756574    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:40.858065 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:40.858029    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:41.358264 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:41.358228    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:41.417416 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:41.417384    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:42.705001 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:42.704946    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:45.037675 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:45.037642    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:45.134112 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:45.134077    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:45.196253 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:45.196213    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:45.834175 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:45.834138    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-h6ncv"]
Sep 08 20:33:45.834475 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:45.834462    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb075287-9192-42ff-9331-28ac0fa29065" containerName="console"
Sep 08 20:33:45.834519 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:45.834476    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb075287-9192-42ff-9331-28ac0fa29065" containerName="console"
Sep 08 20:33:45.834563 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:45.834554    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb075287-9192-42ff-9331-28ac0fa29065" containerName="console"
Sep 08 20:33:45.838586 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:45.838569    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-h6ncv"
Sep 08 20:33:45.840862 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:45.840838    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 08 20:33:45.841018 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:45.840947    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 08 20:33:45.841817 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:45.841798    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-l78c9\""
Sep 08 20:33:45.844185 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:45.844163    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-h6ncv"]
Sep 08 20:33:45.876423 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:45.876387    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:45.983649 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:45.983616    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:45.987600 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:45.987575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5c8x\" (UniqueName: \"kubernetes.io/projected/70c0edef-ebb0-4221-bc49-a01bdda58b1f-kube-api-access-l5c8x\") pod \"authorino-7bcfd8f9cd-h6ncv\" (UID: \"70c0edef-ebb0-4221-bc49-a01bdda58b1f\") " pod="kuadrant-system/authorino-7bcfd8f9cd-h6ncv"
Sep 08 20:33:46.076866 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:46.076836    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:46.088344 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:46.088276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5c8x\" (UniqueName: \"kubernetes.io/projected/70c0edef-ebb0-4221-bc49-a01bdda58b1f-kube-api-access-l5c8x\") pod \"authorino-7bcfd8f9cd-h6ncv\" (UID: \"70c0edef-ebb0-4221-bc49-a01bdda58b1f\") " pod="kuadrant-system/authorino-7bcfd8f9cd-h6ncv"
Sep 08 20:33:46.096731 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:46.096701    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5c8x\" (UniqueName: \"kubernetes.io/projected/70c0edef-ebb0-4221-bc49-a01bdda58b1f-kube-api-access-l5c8x\") pod \"authorino-7bcfd8f9cd-h6ncv\" (UID: \"70c0edef-ebb0-4221-bc49-a01bdda58b1f\") " pod="kuadrant-system/authorino-7bcfd8f9cd-h6ncv"
Sep 08 20:33:46.148595 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:46.148558    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-h6ncv"
Sep 08 20:33:46.416900 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:46.416828    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-h6ncv"]
Sep 08 20:33:46.420556 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:33:46.420524    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70c0edef_ebb0_4221_bc49_a01bdda58b1f.slice/crio-016d3cb33d14262847d55cfd221f67e300a73928b804bbd43ad2b969fb3da6bc WatchSource:0}: Error finding container 016d3cb33d14262847d55cfd221f67e300a73928b804bbd43ad2b969fb3da6bc: Status 404 returned error can't find the container with id 016d3cb33d14262847d55cfd221f67e300a73928b804bbd43ad2b969fb3da6bc
Sep 08 20:33:47.124349 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:47.124309    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-h6ncv" event={"ID":"70c0edef-ebb0-4221-bc49-a01bdda58b1f","Type":"ContainerStarted","Data":"016d3cb33d14262847d55cfd221f67e300a73928b804bbd43ad2b969fb3da6bc"}
Sep 08 20:33:49.136096 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:49.136057    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-h6ncv" event={"ID":"70c0edef-ebb0-4221-bc49-a01bdda58b1f","Type":"ContainerStarted","Data":"c7e952dff9dff89a253bbc75c12a26da8a5863e0b212a4611dddba93c9e14be7"}
Sep 08 20:33:49.153690 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:49.153639    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-h6ncv" podStartSLOduration=1.602679191 podStartE2EDuration="4.153623152s" totalImagesPullingTime="2.550943961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:33:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:33:49.152690879 +0000 UTC m=+502.699940361" watchObservedRunningTime="2026-09-08 20:33:49.153623152 +0000 UTC m=+502.700872633"
Sep 08 20:33:50.759835 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:50.759803    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:50.860499 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:50.860470    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:51.162589 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:51.162552    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:51.263114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:51.263082    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:51.565331 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:51.565245    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:51.665943 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:51.665912    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:53.303222 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:53.303184    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:53.406080 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:53.406046    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:57.963816 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:57.963777    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:33:58.064094 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:33:58.064058    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:03.150626 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:03.150585    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-h6ncv"]
Sep 08 20:34:03.151135 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:03.150761    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-h6ncv" podUID="70c0edef-ebb0-4221-bc49-a01bdda58b1f" containerName="authorino" containerID="cri-o://c7e952dff9dff89a253bbc75c12a26da8a5863e0b212a4611dddba93c9e14be7" gracePeriod=30
Sep 08 20:34:03.395946 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:03.395922    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-h6ncv"
Sep 08 20:34:03.432631 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:03.432551    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5c8x\" (UniqueName: \"kubernetes.io/projected/70c0edef-ebb0-4221-bc49-a01bdda58b1f-kube-api-access-l5c8x\") pod \"70c0edef-ebb0-4221-bc49-a01bdda58b1f\" (UID: \"70c0edef-ebb0-4221-bc49-a01bdda58b1f\") "
Sep 08 20:34:03.434752 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:03.434716    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c0edef-ebb0-4221-bc49-a01bdda58b1f-kube-api-access-l5c8x" (OuterVolumeSpecName: "kube-api-access-l5c8x") pod "70c0edef-ebb0-4221-bc49-a01bdda58b1f" (UID: "70c0edef-ebb0-4221-bc49-a01bdda58b1f"). InnerVolumeSpecName "kube-api-access-l5c8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:34:03.534124 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:03.534089    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5c8x\" (UniqueName: \"kubernetes.io/projected/70c0edef-ebb0-4221-bc49-a01bdda58b1f-kube-api-access-l5c8x\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:34:04.174670 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:04.174637    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:04.203501 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:04.203466    2580 generic.go:356] "Generic (PLEG): container finished" podID="70c0edef-ebb0-4221-bc49-a01bdda58b1f" containerID="c7e952dff9dff89a253bbc75c12a26da8a5863e0b212a4611dddba93c9e14be7" exitCode=0
Sep 08 20:34:04.203665 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:04.203505    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-h6ncv"
Sep 08 20:34:04.203665 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:04.203519    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-h6ncv" event={"ID":"70c0edef-ebb0-4221-bc49-a01bdda58b1f","Type":"ContainerDied","Data":"c7e952dff9dff89a253bbc75c12a26da8a5863e0b212a4611dddba93c9e14be7"}
Sep 08 20:34:04.203665 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:04.203572    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-h6ncv" event={"ID":"70c0edef-ebb0-4221-bc49-a01bdda58b1f","Type":"ContainerDied","Data":"016d3cb33d14262847d55cfd221f67e300a73928b804bbd43ad2b969fb3da6bc"}
Sep 08 20:34:04.203665 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:04.203592    2580 scope.go:117] "RemoveContainer" containerID="c7e952dff9dff89a253bbc75c12a26da8a5863e0b212a4611dddba93c9e14be7"
Sep 08 20:34:04.212860 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:04.212842    2580 scope.go:117] "RemoveContainer" containerID="c7e952dff9dff89a253bbc75c12a26da8a5863e0b212a4611dddba93c9e14be7"
Sep 08 20:34:04.213145 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:34:04.213128    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7e952dff9dff89a253bbc75c12a26da8a5863e0b212a4611dddba93c9e14be7\": container with ID starting with c7e952dff9dff89a253bbc75c12a26da8a5863e0b212a4611dddba93c9e14be7 not found: ID does not exist" containerID="c7e952dff9dff89a253bbc75c12a26da8a5863e0b212a4611dddba93c9e14be7"
Sep 08 20:34:04.213200 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:04.213154    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7e952dff9dff89a253bbc75c12a26da8a5863e0b212a4611dddba93c9e14be7"} err="failed to get container status \"c7e952dff9dff89a253bbc75c12a26da8a5863e0b212a4611dddba93c9e14be7\": rpc error: code = NotFound desc = could not find container \"c7e952dff9dff89a253bbc75c12a26da8a5863e0b212a4611dddba93c9e14be7\": container with ID starting with c7e952dff9dff89a253bbc75c12a26da8a5863e0b212a4611dddba93c9e14be7 not found: ID does not exist"
Sep 08 20:34:04.226394 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:04.226364    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-h6ncv"]
Sep 08 20:34:04.230915 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:04.230893    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-h6ncv"]
Sep 08 20:34:04.244582 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:04.244554    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:04.923605 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:04.923575    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70c0edef-ebb0-4221-bc49-a01bdda58b1f" path="/var/lib/kubelet/pods/70c0edef-ebb0-4221-bc49-a01bdda58b1f/volumes"
Sep 08 20:34:06.220398 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:06.220358    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:06.321218 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:06.321184    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:06.708323 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:06.708287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:07.172740 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:07.172703    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:07.263902 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:07.263863    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:16.496540 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:16.496495    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:16.672434 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:16.672384    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:16.711794 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:16.711760    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-x2x9d"]
Sep 08 20:34:16.712112 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:16.712100    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c0edef-ebb0-4221-bc49-a01bdda58b1f" containerName="authorino"
Sep 08 20:34:16.712162 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:16.712114    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c0edef-ebb0-4221-bc49-a01bdda58b1f" containerName="authorino"
Sep 08 20:34:16.712197 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:16.712183    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c0edef-ebb0-4221-bc49-a01bdda58b1f" containerName="authorino"
Sep 08 20:34:16.715414 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:16.715395    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-x2x9d"
Sep 08 20:34:16.717340 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:16.717314    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-l78c9\""
Sep 08 20:34:16.717427 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:16.717326    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 08 20:34:16.717812 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:16.717798    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 08 20:34:16.720926 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:16.720907    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-x2x9d"]
Sep 08 20:34:16.789331 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:16.789244    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-x2x9d"]
Sep 08 20:34:16.789540 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:34:16.789520    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-nvccq], unattached volumes=[], failed to process volumes=[kube-api-access-nvccq]: context canceled" pod="kuadrant-system/authorino-557f876f69-x2x9d" podUID="0fe6883a-5ded-48ee-9578-57edcb07946b"
Sep 08 20:34:16.855841 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:16.855802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvccq\" (UniqueName: \"kubernetes.io/projected/0fe6883a-5ded-48ee-9578-57edcb07946b-kube-api-access-nvccq\") pod \"authorino-557f876f69-x2x9d\" (UID: \"0fe6883a-5ded-48ee-9578-57edcb07946b\") " pod="kuadrant-system/authorino-557f876f69-x2x9d"
Sep 08 20:34:16.956667 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:16.956631    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvccq\" (UniqueName: \"kubernetes.io/projected/0fe6883a-5ded-48ee-9578-57edcb07946b-kube-api-access-nvccq\") pod \"authorino-557f876f69-x2x9d\" (UID: \"0fe6883a-5ded-48ee-9578-57edcb07946b\") " pod="kuadrant-system/authorino-557f876f69-x2x9d"
Sep 08 20:34:16.969162 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:16.969129    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvccq\" (UniqueName: \"kubernetes.io/projected/0fe6883a-5ded-48ee-9578-57edcb07946b-kube-api-access-nvccq\") pod \"authorino-557f876f69-x2x9d\" (UID: \"0fe6883a-5ded-48ee-9578-57edcb07946b\") " pod="kuadrant-system/authorino-557f876f69-x2x9d"
Sep 08 20:34:17.263284 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:17.263248    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-x2x9d"
Sep 08 20:34:17.269424 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:17.269400    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-x2x9d"
Sep 08 20:34:17.360006 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:17.359941    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvccq\" (UniqueName: \"kubernetes.io/projected/0fe6883a-5ded-48ee-9578-57edcb07946b-kube-api-access-nvccq\") pod \"0fe6883a-5ded-48ee-9578-57edcb07946b\" (UID: \"0fe6883a-5ded-48ee-9578-57edcb07946b\") "
Sep 08 20:34:17.362019 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:17.361989    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fe6883a-5ded-48ee-9578-57edcb07946b-kube-api-access-nvccq" (OuterVolumeSpecName: "kube-api-access-nvccq") pod "0fe6883a-5ded-48ee-9578-57edcb07946b" (UID: "0fe6883a-5ded-48ee-9578-57edcb07946b"). InnerVolumeSpecName "kube-api-access-nvccq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:34:17.461516 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:17.461474    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvccq\" (UniqueName: \"kubernetes.io/projected/0fe6883a-5ded-48ee-9578-57edcb07946b-kube-api-access-nvccq\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:34:18.268411 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:18.268382    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-x2x9d"
Sep 08 20:34:18.295501 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:18.295445    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-x2x9d"]
Sep 08 20:34:18.299992 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:18.299154    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-x2x9d"]
Sep 08 20:34:18.923321 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:18.923267    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fe6883a-5ded-48ee-9578-57edcb07946b" path="/var/lib/kubelet/pods/0fe6883a-5ded-48ee-9578-57edcb07946b/volumes"
Sep 08 20:34:22.986033 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:22.986002    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:23.080270 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:23.080233    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:23.179980 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:23.179917    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:23.348280 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:23.348207    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:26.040446 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:26.040410    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:26.139124 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:26.139094    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:58.448811 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:58.448774    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:58.553565 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:58.553526    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:34:59.907797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:34:59.907764    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:35:00.007134 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.007099    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:35:00.107325 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.107287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:35:00.138675 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.138640    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"]
Sep 08 20:35:00.141070 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.141048    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 20:35:00.143795 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.143771    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 08 20:35:00.143918 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.143794    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 08 20:35:00.143918 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.143804    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 20:35:00.143918 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.143834    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 08 20:35:00.144212 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.144196    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z8244\""
Sep 08 20:35:00.152872 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.152844    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"]
Sep 08 20:35:00.176238 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.176156    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:35:00.215538 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.215499    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b66hs\" (UniqueName: \"kubernetes.io/projected/121a28e5-e356-4205-8c21-2ac12d541697-kube-api-access-b66hs\") pod \"maas-api-6b96cdc7c6-hm2sn\" (UID: \"121a28e5-e356-4205-8c21-2ac12d541697\") " pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 20:35:00.215717 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.215555    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/121a28e5-e356-4205-8c21-2ac12d541697-openshift-service-ca\") pod \"maas-api-6b96cdc7c6-hm2sn\" (UID: \"121a28e5-e356-4205-8c21-2ac12d541697\") " pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 20:35:00.215717 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.215623    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/121a28e5-e356-4205-8c21-2ac12d541697-metrics-certs\") pod \"maas-api-6b96cdc7c6-hm2sn\" (UID: \"121a28e5-e356-4205-8c21-2ac12d541697\") " pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 20:35:00.215717 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.215667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/121a28e5-e356-4205-8c21-2ac12d541697-maas-api-tls\") pod \"maas-api-6b96cdc7c6-hm2sn\" (UID: \"121a28e5-e356-4205-8c21-2ac12d541697\") " pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 20:35:00.316986 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.316923    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b66hs\" (UniqueName: \"kubernetes.io/projected/121a28e5-e356-4205-8c21-2ac12d541697-kube-api-access-b66hs\") pod \"maas-api-6b96cdc7c6-hm2sn\" (UID: \"121a28e5-e356-4205-8c21-2ac12d541697\") " pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 20:35:00.317163 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.317005    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/121a28e5-e356-4205-8c21-2ac12d541697-openshift-service-ca\") pod \"maas-api-6b96cdc7c6-hm2sn\" (UID: \"121a28e5-e356-4205-8c21-2ac12d541697\") " pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 20:35:00.317163 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.317031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/121a28e5-e356-4205-8c21-2ac12d541697-metrics-certs\") pod \"maas-api-6b96cdc7c6-hm2sn\" (UID: \"121a28e5-e356-4205-8c21-2ac12d541697\") " pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 20:35:00.317163 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.317062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/121a28e5-e356-4205-8c21-2ac12d541697-maas-api-tls\") pod \"maas-api-6b96cdc7c6-hm2sn\" (UID: \"121a28e5-e356-4205-8c21-2ac12d541697\") " pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 20:35:00.317688 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.317660    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/121a28e5-e356-4205-8c21-2ac12d541697-openshift-service-ca\") pod \"maas-api-6b96cdc7c6-hm2sn\" (UID: \"121a28e5-e356-4205-8c21-2ac12d541697\") " pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 20:35:00.319483 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.319462    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/121a28e5-e356-4205-8c21-2ac12d541697-metrics-certs\") pod \"maas-api-6b96cdc7c6-hm2sn\" (UID: \"121a28e5-e356-4205-8c21-2ac12d541697\") " pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 20:35:00.319614 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.319545    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/121a28e5-e356-4205-8c21-2ac12d541697-maas-api-tls\") pod \"maas-api-6b96cdc7c6-hm2sn\" (UID: \"121a28e5-e356-4205-8c21-2ac12d541697\") " pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 20:35:00.324568 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.324545    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b66hs\" (UniqueName: \"kubernetes.io/projected/121a28e5-e356-4205-8c21-2ac12d541697-kube-api-access-b66hs\") pod \"maas-api-6b96cdc7c6-hm2sn\" (UID: \"121a28e5-e356-4205-8c21-2ac12d541697\") " pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 20:35:00.451598 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.451502    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 20:35:00.536712 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.536677    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:35:00.647018 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.646980    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:35:00.723871 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:00.723791    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"]
Sep 08 20:35:00.727410 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:35:00.727386    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod121a28e5_e356_4205_8c21_2ac12d541697.slice/crio-5841d5e8f1c77abc796d21439b8cf7adbf458471a337d1b8785c587a6618639f WatchSource:0}: Error finding container 5841d5e8f1c77abc796d21439b8cf7adbf458471a337d1b8785c587a6618639f: Status 404 returned error can't find the container with id 5841d5e8f1c77abc796d21439b8cf7adbf458471a337d1b8785c587a6618639f
Sep 08 20:35:01.463530 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:01.463496    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn" event={"ID":"121a28e5-e356-4205-8c21-2ac12d541697","Type":"ContainerStarted","Data":"5841d5e8f1c77abc796d21439b8cf7adbf458471a337d1b8785c587a6618639f"}
Sep 08 20:35:03.473992 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:03.473941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn" event={"ID":"121a28e5-e356-4205-8c21-2ac12d541697","Type":"ContainerStarted","Data":"6e1d5bd46a76d3e37516b1ab211392960e090a208c9a918ecb2501b110d811c3"}
Sep 08 20:35:03.474380 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:03.474032    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 20:35:03.488707 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:03.488648    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn" podStartSLOduration=1.751362748 podStartE2EDuration="3.488631617s" totalImagesPullingTime="1.737268869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:35:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:35:03.487375927 +0000 UTC m=+577.034625407" watchObservedRunningTime="2026-09-08 20:35:03.488631617 +0000 UTC m=+577.035881097"
Sep 08 20:35:09.483475 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:09.483444    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 20:35:10.523201 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:10.523169    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:35:10.674697 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:10.674664    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:35:16.352396 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:16.352355    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:35:24.398930 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.398891    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"]
Sep 08 20:35:24.402117 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.402093    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.405079 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.405055    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xsfxp\""
Sep 08 20:35:24.405212 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.405104    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 08 20:35:24.405212 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.405132    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 08 20:35:24.405328 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.405076    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 08 20:35:24.411669 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.411642    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"]
Sep 08 20:35:24.431631 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.431601    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:35:24.535972 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.535920    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59008731-c2ba-4a82-a842-aea570790e12-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.536144 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.535992    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59008731-c2ba-4a82-a842-aea570790e12-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.536144 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.536030    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59008731-c2ba-4a82-a842-aea570790e12-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.536144 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.536048    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59008731-c2ba-4a82-a842-aea570790e12-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.536144 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.536129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hjwt\" (UniqueName: \"kubernetes.io/projected/59008731-c2ba-4a82-a842-aea570790e12-kube-api-access-7hjwt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.536271 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.536186    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59008731-c2ba-4a82-a842-aea570790e12-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.637460 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.637416    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59008731-c2ba-4a82-a842-aea570790e12-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.637460 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.637465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59008731-c2ba-4a82-a842-aea570790e12-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.637685 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.637512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59008731-c2ba-4a82-a842-aea570790e12-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.637685 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.637535    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59008731-c2ba-4a82-a842-aea570790e12-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.637685 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.637576    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hjwt\" (UniqueName: \"kubernetes.io/projected/59008731-c2ba-4a82-a842-aea570790e12-kube-api-access-7hjwt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.637685 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.637629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59008731-c2ba-4a82-a842-aea570790e12-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.638067 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.638037    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59008731-c2ba-4a82-a842-aea570790e12-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.638184 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.638068    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59008731-c2ba-4a82-a842-aea570790e12-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.638184 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.638096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59008731-c2ba-4a82-a842-aea570790e12-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.639799 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.639775    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59008731-c2ba-4a82-a842-aea570790e12-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.640018 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.639998    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59008731-c2ba-4a82-a842-aea570790e12-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.645784 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.645761    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hjwt\" (UniqueName: \"kubernetes.io/projected/59008731-c2ba-4a82-a842-aea570790e12-kube-api-access-7hjwt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nn7g8\" (UID: \"59008731-c2ba-4a82-a842-aea570790e12\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.714249 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.714159    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:24.985824 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:24.985798    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"]
Sep 08 20:35:24.988539 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:35:24.988509    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59008731_c2ba_4a82_a842_aea570790e12.slice/crio-e367488f5194bd51abd80201c6ff97d0c56eff121ac768d7703a5c7ede965ba6 WatchSource:0}: Error finding container e367488f5194bd51abd80201c6ff97d0c56eff121ac768d7703a5c7ede965ba6: Status 404 returned error can't find the container with id e367488f5194bd51abd80201c6ff97d0c56eff121ac768d7703a5c7ede965ba6
Sep 08 20:35:25.578041 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:25.577997    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8" event={"ID":"59008731-c2ba-4a82-a842-aea570790e12","Type":"ContainerStarted","Data":"e367488f5194bd51abd80201c6ff97d0c56eff121ac768d7703a5c7ede965ba6"}
Sep 08 20:35:26.584476 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:26.584442    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8" event={"ID":"59008731-c2ba-4a82-a842-aea570790e12","Type":"ContainerStarted","Data":"22241c4f7195a40cb1d42f5a7a52df24d9bc321ab97836600ce073d5e29b3ff4"}
Sep 08 20:35:26.605381 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:26.605324    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8" podStartSLOduration=1.653406836 podStartE2EDuration="2.605307195s" totalImagesPullingTime="951.900359ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:35:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:35:26.603062923 +0000 UTC m=+600.150312404" watchObservedRunningTime="2026-09-08 20:35:26.605307195 +0000 UTC m=+600.152556675"
Sep 08 20:35:29.781601 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:29.781563    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:35:34.715316 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:34.715272    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:34.715751 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:34.715611    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:34.727946 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:34.727914    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:35.636769 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:35.636722    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nn7g8"
Sep 08 20:35:39.684526 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:39.684488    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:35:49.234924 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:49.234835    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:35:51.586149 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:51.586111    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:35:52.506056 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:35:52.506015    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:36:11.234648 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:11.234613    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:36:22.890425 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:22.890390    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"]
Sep 08 20:36:22.893806 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:22.893786    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:22.896152 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:22.896133    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 08 20:36:22.902156 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:22.902134    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"]
Sep 08 20:36:22.913217 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:22.913194    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:36:23.057523 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.057489    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.057523 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.057530    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl4ll\" (UniqueName: \"kubernetes.io/projected/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-kube-api-access-bl4ll\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.057523 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.057571    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.057809 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.057700    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.057809 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.057750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.057872 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.057830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.158770 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.158653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl4ll\" (UniqueName: \"kubernetes.io/projected/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-kube-api-access-bl4ll\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.158770 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.158723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.158770 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.158756    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.158770 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.158777    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.159145 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.158823    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.159145 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.158931    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.159320 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.159284    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.159425 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.159292    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.159425 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.159330    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.161133 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.161112    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.161300 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.161282    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.166876 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.166854    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl4ll\" (UniqueName: \"kubernetes.io/projected/0f7449da-bd90-4bd6-b14b-3953c3fb0cf1-kube-api-access-bl4ll\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-t7v26\" (UID: \"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.203977 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.203931    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:23.484878 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.484851    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"]
Sep 08 20:36:23.486890 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:36:23.486863    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f7449da_bd90_4bd6_b14b_3953c3fb0cf1.slice/crio-952175c90a6ce9d729dfd991de4419fef5ec5b9bd3a03d36b997aa9bc2c8d934 WatchSource:0}: Error finding container 952175c90a6ce9d729dfd991de4419fef5ec5b9bd3a03d36b997aa9bc2c8d934: Status 404 returned error can't find the container with id 952175c90a6ce9d729dfd991de4419fef5ec5b9bd3a03d36b997aa9bc2c8d934
Sep 08 20:36:23.488174 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.488154    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 20:36:23.847075 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.846979    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26" event={"ID":"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1","Type":"ContainerStarted","Data":"0492ceee34cf9acc6edffbb26eae880153499b8da16093213b63e2d39d529fc9"}
Sep 08 20:36:23.847075 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.847023    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26" event={"ID":"0f7449da-bd90-4bd6-b14b-3953c3fb0cf1","Type":"ContainerStarted","Data":"952175c90a6ce9d729dfd991de4419fef5ec5b9bd3a03d36b997aa9bc2c8d934"}
Sep 08 20:36:23.864682 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:23.864623    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26" podStartSLOduration=1.6788661070000002 podStartE2EDuration="1.864602569s" totalImagesPullingTime="185.736462ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:36:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:36:23.862791939 +0000 UTC m=+657.410041410" watchObservedRunningTime="2026-09-08 20:36:23.864602569 +0000 UTC m=+657.411852050"
Sep 08 20:36:33.204650 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:33.204609    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:33.205160 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:33.204686    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:33.217097 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:33.217074    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:36:33.902922 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:36:33.902892    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-t7v26"
Sep 08 20:37:30.622242 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:30.622114    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:37:58.500512 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:58.500474    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b8c6c7bc8-mg4qh"]
Sep 08 20:37:58.503692 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:58.503676    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b8c6c7bc8-mg4qh"
Sep 08 20:37:58.506264 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:58.506237    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-l78c9\""
Sep 08 20:37:58.506383 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:58.506318    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 08 20:37:58.506451 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:58.506383    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 08 20:37:58.506451 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:58.506393    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 08 20:37:58.510104 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:58.510081    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b8c6c7bc8-mg4qh"]
Sep 08 20:37:58.540755 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:58.540727    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e3627210-e1cc-4535-ab72-c888e373bc65-tls-cert\") pod \"authorino-b8c6c7bc8-mg4qh\" (UID: \"e3627210-e1cc-4535-ab72-c888e373bc65\") " pod="kuadrant-system/authorino-b8c6c7bc8-mg4qh"
Sep 08 20:37:58.540927 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:58.540764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62wrv\" (UniqueName: \"kubernetes.io/projected/e3627210-e1cc-4535-ab72-c888e373bc65-kube-api-access-62wrv\") pod \"authorino-b8c6c7bc8-mg4qh\" (UID: \"e3627210-e1cc-4535-ab72-c888e373bc65\") " pod="kuadrant-system/authorino-b8c6c7bc8-mg4qh"
Sep 08 20:37:58.642126 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:58.642089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e3627210-e1cc-4535-ab72-c888e373bc65-tls-cert\") pod \"authorino-b8c6c7bc8-mg4qh\" (UID: \"e3627210-e1cc-4535-ab72-c888e373bc65\") " pod="kuadrant-system/authorino-b8c6c7bc8-mg4qh"
Sep 08 20:37:58.642297 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:58.642131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62wrv\" (UniqueName: \"kubernetes.io/projected/e3627210-e1cc-4535-ab72-c888e373bc65-kube-api-access-62wrv\") pod \"authorino-b8c6c7bc8-mg4qh\" (UID: \"e3627210-e1cc-4535-ab72-c888e373bc65\") " pod="kuadrant-system/authorino-b8c6c7bc8-mg4qh"
Sep 08 20:37:58.644631 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:58.644597    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e3627210-e1cc-4535-ab72-c888e373bc65-tls-cert\") pod \"authorino-b8c6c7bc8-mg4qh\" (UID: \"e3627210-e1cc-4535-ab72-c888e373bc65\") " pod="kuadrant-system/authorino-b8c6c7bc8-mg4qh"
Sep 08 20:37:58.650178 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:58.650155    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62wrv\" (UniqueName: \"kubernetes.io/projected/e3627210-e1cc-4535-ab72-c888e373bc65-kube-api-access-62wrv\") pod \"authorino-b8c6c7bc8-mg4qh\" (UID: \"e3627210-e1cc-4535-ab72-c888e373bc65\") " pod="kuadrant-system/authorino-b8c6c7bc8-mg4qh"
Sep 08 20:37:58.814669 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:58.814583    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b8c6c7bc8-mg4qh"
Sep 08 20:37:59.096202 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:59.096127    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b8c6c7bc8-mg4qh"]
Sep 08 20:37:59.099763 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:37:59.099740    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3627210_e1cc_4535_ab72_c888e373bc65.slice/crio-de56ad5940e88ef67e4b064af78587a7cff9e1d14557a7da167690b0e5785be8 WatchSource:0}: Error finding container de56ad5940e88ef67e4b064af78587a7cff9e1d14557a7da167690b0e5785be8: Status 404 returned error can't find the container with id de56ad5940e88ef67e4b064af78587a7cff9e1d14557a7da167690b0e5785be8
Sep 08 20:37:59.283774 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:37:59.283734    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b8c6c7bc8-mg4qh" event={"ID":"e3627210-e1cc-4535-ab72-c888e373bc65","Type":"ContainerStarted","Data":"de56ad5940e88ef67e4b064af78587a7cff9e1d14557a7da167690b0e5785be8"}
Sep 08 20:38:00.290947 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:00.290905    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b8c6c7bc8-mg4qh" event={"ID":"e3627210-e1cc-4535-ab72-c888e373bc65","Type":"ContainerStarted","Data":"dc90ccad44357dd1912c3de06a8af6635243d1f41fd360b85ff52080c29445d7"}
Sep 08 20:38:00.309050 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:00.308984    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b8c6c7bc8-mg4qh" podStartSLOduration=1.676075844 podStartE2EDuration="2.30894505s" totalImagesPullingTime="632.869206ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:37:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:38:00.308468544 +0000 UTC m=+753.855718037" watchObservedRunningTime="2026-09-08 20:38:00.30894505 +0000 UTC m=+753.856194532"
Sep 08 20:38:00.843321 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:00.843279    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:00.933562 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:00.933525    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:01.982489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:01.982451    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:02.083133 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:02.083093    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:17.094509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:17.094474    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:17.231582 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:17.231545    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:17.411658 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:17.411618    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:17.913514 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:17.913477    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:18.539478 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:18.539445    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:18.653305 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:18.653268    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:19.513241 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:19.513211    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:20.665661 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:20.665618    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:20.759869 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:20.759831    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:22.117492 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.117455    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"]
Sep 08 20:38:22.127667 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.127636    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:38:22.130382 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.130195    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-6f8575-serving-cert\""
Sep 08 20:38:22.130382 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.130257    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"]
Sep 08 20:38:22.172847 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.172813    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:22.264715 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.264677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p89fd\" (UniqueName: \"kubernetes.io/projected/6b705417-23b4-4c43-a46d-cf5c59b3fc52-kube-api-access-p89fd\") pod \"maas-api-e2e-inf-a-6f8575-5d55458675-tswtl\" (UID: \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:38:22.264715 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.264715    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b705417-23b4-4c43-a46d-cf5c59b3fc52-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6f8575-5d55458675-tswtl\" (UID: \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:38:22.264945 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.264805    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b705417-23b4-4c43-a46d-cf5c59b3fc52-maas-api-tls\") pod \"maas-api-e2e-inf-a-6f8575-5d55458675-tswtl\" (UID: \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:38:22.265004 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.264982    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b705417-23b4-4c43-a46d-cf5c59b3fc52-metrics-certs\") pod \"maas-api-e2e-inf-a-6f8575-5d55458675-tswtl\" (UID: \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:38:22.293869 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.293824    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:22.366307 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.366269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b705417-23b4-4c43-a46d-cf5c59b3fc52-metrics-certs\") pod \"maas-api-e2e-inf-a-6f8575-5d55458675-tswtl\" (UID: \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:38:22.366307 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.366313    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p89fd\" (UniqueName: \"kubernetes.io/projected/6b705417-23b4-4c43-a46d-cf5c59b3fc52-kube-api-access-p89fd\") pod \"maas-api-e2e-inf-a-6f8575-5d55458675-tswtl\" (UID: \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:38:22.366723 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.366333    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b705417-23b4-4c43-a46d-cf5c59b3fc52-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6f8575-5d55458675-tswtl\" (UID: \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:38:22.366723 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.366388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b705417-23b4-4c43-a46d-cf5c59b3fc52-maas-api-tls\") pod \"maas-api-e2e-inf-a-6f8575-5d55458675-tswtl\" (UID: \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:38:22.367159 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.367133    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b705417-23b4-4c43-a46d-cf5c59b3fc52-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6f8575-5d55458675-tswtl\" (UID: \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:38:22.369072 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.369007    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b705417-23b4-4c43-a46d-cf5c59b3fc52-metrics-certs\") pod \"maas-api-e2e-inf-a-6f8575-5d55458675-tswtl\" (UID: \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:38:22.369072 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.369007    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b705417-23b4-4c43-a46d-cf5c59b3fc52-maas-api-tls\") pod \"maas-api-e2e-inf-a-6f8575-5d55458675-tswtl\" (UID: \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:38:22.374278 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.374255    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p89fd\" (UniqueName: \"kubernetes.io/projected/6b705417-23b4-4c43-a46d-cf5c59b3fc52-kube-api-access-p89fd\") pod \"maas-api-e2e-inf-a-6f8575-5d55458675-tswtl\" (UID: \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:38:22.395230 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.395197    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:22.441074 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.441040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:38:22.713016 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.712983    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"]
Sep 08 20:38:22.715232 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:38:22.715196    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b705417_23b4_4c43_a46d_cf5c59b3fc52.slice/crio-25aab8421de6d8d182252dc5dfc04c19a4fe5ba905e82de2312454c9771adfa5 WatchSource:0}: Error finding container 25aab8421de6d8d182252dc5dfc04c19a4fe5ba905e82de2312454c9771adfa5: Status 404 returned error can't find the container with id 25aab8421de6d8d182252dc5dfc04c19a4fe5ba905e82de2312454c9771adfa5
Sep 08 20:38:22.750089 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.750041    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:22.847234 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:22.847197    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:23.402064 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:23.402027    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl" event={"ID":"6b705417-23b4-4c43-a46d-cf5c59b3fc52","Type":"ContainerStarted","Data":"9c5a0acbf7ab2bc0f7b0008cac6698441e1de8246062272e4b72d8069ee51bed"}
Sep 08 20:38:23.402460 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:23.402085    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl" event={"ID":"6b705417-23b4-4c43-a46d-cf5c59b3fc52","Type":"ContainerStarted","Data":"25aab8421de6d8d182252dc5dfc04c19a4fe5ba905e82de2312454c9771adfa5"}
Sep 08 20:38:23.402460 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:23.402222    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:38:23.419375 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:23.419279    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl" podStartSLOduration=0.975323353 podStartE2EDuration="1.419262558s" totalImagesPullingTime="443.939205ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:38:23.418385296 +0000 UTC m=+776.965634771" watchObservedRunningTime="2026-09-08 20:38:23.419262558 +0000 UTC m=+776.966512038"
Sep 08 20:38:25.307623 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.307576    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:25.377625 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.377591    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:25.526100 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.526067    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:25.782742 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.782708    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"]
Sep 08 20:38:25.786293 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.786270    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.788509 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.788488    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c73d714b-openshift-default-dockercfg-vt94z\""
Sep 08 20:38:25.797081 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.797054    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"]
Sep 08 20:38:25.807731 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.807695    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-credential-socket\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.807731 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.807733    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x69bf\" (UniqueName: \"kubernetes.io/projected/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-kube-api-access-x69bf\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.807939 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.807755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-envoy\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.807939 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.807836    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-data\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.807939 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.807876    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-workload-certs\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.807939 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.807938    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-workload-socket\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.808109 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.807981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-podinfo\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.808109 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.808003    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-token\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.808109 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.808025    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istiod-ca-cert\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.816758 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.816719    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:25.908572 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.908530    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-workload-certs\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.908765 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.908587    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-workload-socket\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.908765 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.908627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-podinfo\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.908765 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.908659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-token\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.908765 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.908686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istiod-ca-cert\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.909031 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.908902    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-credential-socket\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.909031 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.908946    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x69bf\" (UniqueName: \"kubernetes.io/projected/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-kube-api-access-x69bf\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.909031 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.908998    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-workload-socket\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.909031 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.909014    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-envoy\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.909242 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.909084    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-workload-certs\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.909242 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.909106    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-data\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.909396 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.909375    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-credential-socket\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.909463 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.909443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istiod-ca-cert\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.909513 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.909446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-data\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.911124 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.911105    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-envoy\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.911235 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.911220    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-podinfo\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.916213 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.916189    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-token\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.916370 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.916350    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x69bf\" (UniqueName: \"kubernetes.io/projected/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-kube-api-access-x69bf\") pod \"e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") " pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:25.922492 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:25.922451    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:26.097813 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.097717    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:26.373010 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.372984    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"]
Sep 08 20:38:26.374919 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:38:26.374893    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2de30cd_cc04_4dec_bfdd_1b2fb23feefc.slice/crio-988708c7e8e9d49d0265a51456a09057937583e4a787ba1cd8707447e8a94d76 WatchSource:0}: Error finding container 988708c7e8e9d49d0265a51456a09057937583e4a787ba1cd8707447e8a94d76: Status 404 returned error can't find the container with id 988708c7e8e9d49d0265a51456a09057937583e4a787ba1cd8707447e8a94d76
Sep 08 20:38:26.377640 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.377610    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:38:26.377736 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.377715    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:38:26.377796 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.377763    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:38:26.418934 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.418898    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf" event={"ID":"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc","Type":"ContainerStarted","Data":"988708c7e8e9d49d0265a51456a09057937583e4a787ba1cd8707447e8a94d76"}
Sep 08 20:38:26.549862 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.549824    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:26.787085 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.787052    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"]
Sep 08 20:38:26.790615 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.790590    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.792887 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.792864    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-91d46f-openshift-default-dockercfg-7m2xh\""
Sep 08 20:38:26.801342 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.801300    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"]
Sep 08 20:38:26.816782 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.816754    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-credential-socket\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.816936 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.816806    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istiod-ca-cert\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.816936 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.816860    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-podinfo\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.816936 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.816911    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-workload-socket\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.817140 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.816941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-workload-certs\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.817140 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.816973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-envoy\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.817140 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.817016    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6slw\" (UniqueName: \"kubernetes.io/projected/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-kube-api-access-z6slw\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.817140 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.817075    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-data\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.817140 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.817098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-token\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.819855 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.819797    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:26.917810 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.917781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-workload-certs\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.917994 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.917820    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-envoy\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.917994 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.917880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6slw\" (UniqueName: \"kubernetes.io/projected/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-kube-api-access-z6slw\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.917994 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.917909    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-data\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.918265 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.918236    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-workload-certs\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.918411 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.918269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-token\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.918557 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.918528    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-credential-socket\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.918647 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.918625    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istiod-ca-cert\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.918701 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.918673    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-podinfo\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.918752 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.918717    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-workload-socket\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.919265 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.919055    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-credential-socket\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.919356 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.919330    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-workload-socket\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.919420 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.918451    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-data\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.919481 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.919438    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istiod-ca-cert\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.920760 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.920733    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-envoy\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.922041 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.922018    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-podinfo\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.926104 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.926080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-token\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:26.926185 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:26.926154    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6slw\" (UniqueName: \"kubernetes.io/projected/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-kube-api-access-z6slw\") pod \"e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") " pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:27.054023 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:27.053931    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:27.105412 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:27.105379    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-91d46f-openshift-default-dockercfg-7m2xh\""
Sep 08 20:38:27.114230 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:27.114163    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:27.390190 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:27.390164    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"]
Sep 08 20:38:27.392520 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:38:27.392487    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49dbefb7_1b13_4c42_9a1e_dfdf5b5461a4.slice/crio-33a4d9652212658650aa7777c7b0257a91456563cf06ee7f1e1b79e43ad5c778 WatchSource:0}: Error finding container 33a4d9652212658650aa7777c7b0257a91456563cf06ee7f1e1b79e43ad5c778: Status 404 returned error can't find the container with id 33a4d9652212658650aa7777c7b0257a91456563cf06ee7f1e1b79e43ad5c778
Sep 08 20:38:27.401005 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:27.395164    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:38:27.401005 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:27.395418    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:38:27.401005 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:27.395534    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:38:27.425767 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:27.425734    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf" event={"ID":"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc","Type":"ContainerStarted","Data":"77eaf5ce88dae704fc81e12fc9be79632ac7f53da1cabbb177341cacbc001d9b"}
Sep 08 20:38:27.427057 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:27.427028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g" event={"ID":"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4","Type":"ContainerStarted","Data":"33a4d9652212658650aa7777c7b0257a91456563cf06ee7f1e1b79e43ad5c778"}
Sep 08 20:38:27.444979 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:27.444911    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf" podStartSLOduration=2.444897864 podStartE2EDuration="2.444897864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:38:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:38:27.443548237 +0000 UTC m=+780.990797731" watchObservedRunningTime="2026-09-08 20:38:27.444897864 +0000 UTC m=+780.992147344"
Sep 08 20:38:28.098943 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:28.098903    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:28.104371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:28.104344    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:28.433729 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:28.433693    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g" event={"ID":"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4","Type":"ContainerStarted","Data":"2383cf4e77fcf25ed943ede723c62330411140e5bac1370c504e3a4bda9149dd"}
Sep 08 20:38:28.434199 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:28.433776    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:28.434733 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:28.434715    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:28.452388 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:28.452076    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g" podStartSLOduration=2.452054874 podStartE2EDuration="2.452054874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:38:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:38:28.4516851 +0000 UTC m=+781.998934582" watchObservedRunningTime="2026-09-08 20:38:28.452054874 +0000 UTC m=+781.999304355"
Sep 08 20:38:28.811850 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:28.811749    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:28.937291 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:28.937245    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:29.114490 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:29.114392    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:29.124473 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:29.124438    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:29.414069 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:29.414039    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:38:29.439133 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:29.439102    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:29.439920 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:29.439900    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:30.186376 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:30.186342    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:30.267311 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:30.267279    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:30.375281 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:30.375249    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:30.476519 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:30.476441    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:30.577385 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:30.577350    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:30.849469 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:30.849388    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:30.945319 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:30.945284    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:31.934860 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:31.934827    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"]
Sep 08 20:38:31.939447 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:31.939427    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:38:31.946829 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:31.946518    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-91d46f-serving-cert\""
Sep 08 20:38:31.959619 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:31.959572    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"]
Sep 08 20:38:31.966652 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:31.966622    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-maas-api-tls\") pod \"maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht\" (UID: \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:38:31.966799 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:31.966656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-openshift-service-ca\") pod \"maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht\" (UID: \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:38:31.966799 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:31.966725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-metrics-certs\") pod \"maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht\" (UID: \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:38:31.966799 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:31.966757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkrbq\" (UniqueName: \"kubernetes.io/projected/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-kube-api-access-mkrbq\") pod \"maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht\" (UID: \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:38:31.970770 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:31.970746    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:32.067363 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:32.067325    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkrbq\" (UniqueName: \"kubernetes.io/projected/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-kube-api-access-mkrbq\") pod \"maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht\" (UID: \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:38:32.067540 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:32.067430    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-maas-api-tls\") pod \"maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht\" (UID: \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:38:32.067540 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:32.067451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-openshift-service-ca\") pod \"maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht\" (UID: \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:38:32.067540 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:32.067491    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-metrics-certs\") pod \"maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht\" (UID: \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:38:32.068247 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:32.068216    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-openshift-service-ca\") pod \"maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht\" (UID: \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:38:32.070109 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:32.070080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-maas-api-tls\") pod \"maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht\" (UID: \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:38:32.070109 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:32.070095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-metrics-certs\") pod \"maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht\" (UID: \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:38:32.071109 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:32.071081    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:32.075048 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:32.075027    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkrbq\" (UniqueName: \"kubernetes.io/projected/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-kube-api-access-mkrbq\") pod \"maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht\" (UID: \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:38:32.136792 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:32.136752    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:32.250918 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:32.250834    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:38:32.484058 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:32.484010    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:32.527051 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:32.527026    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"]
Sep 08 20:38:32.529540 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:38:32.529510    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfa8b41f_4336_4ba5_a6af_bcdf15f75161.slice/crio-d4c40d97b9ba613a4ded1f0192a29ad8ba6b5247528732828b22ce8c0f710ead WatchSource:0}: Error finding container d4c40d97b9ba613a4ded1f0192a29ad8ba6b5247528732828b22ce8c0f710ead: Status 404 returned error can't find the container with id d4c40d97b9ba613a4ded1f0192a29ad8ba6b5247528732828b22ce8c0f710ead
Sep 08 20:38:32.582280 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:32.582244    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:32.960343 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:32.960309    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:33.460668 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:33.460627    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht" event={"ID":"dfa8b41f-4336-4ba5-a6af-bcdf15f75161","Type":"ContainerStarted","Data":"9a5a750f626d324e56d6efa1b31bb2f0947b2217ec804e90668eaa271dbb33d0"}
Sep 08 20:38:33.460863 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:33.460681    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht" event={"ID":"dfa8b41f-4336-4ba5-a6af-bcdf15f75161","Type":"ContainerStarted","Data":"d4c40d97b9ba613a4ded1f0192a29ad8ba6b5247528732828b22ce8c0f710ead"}
Sep 08 20:38:33.460863 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:33.460779    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:38:33.479299 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:33.479241    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht" podStartSLOduration=1.632077706 podStartE2EDuration="2.479223237s" totalImagesPullingTime="847.145531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:38:33.477241739 +0000 UTC m=+787.024491221" watchObservedRunningTime="2026-09-08 20:38:33.479223237 +0000 UTC m=+787.026472718"
Sep 08 20:38:33.825521 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:33.825426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:36.402213 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:36.402178    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:36.503396 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:36.503350    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:37.963701 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:37.963656    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:38.114028 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:38.113991    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:38.209568 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:38.209531    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:38.311753 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:38.311660    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:39.473370 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:39.473340    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:38:40.959829 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:40.959776    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:42.567157 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:42.567085    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:42.664147 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:42.664104    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:42.770337 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:42.770277    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:43.244029 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:43.243880    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:43.309895 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:43.309857    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:44.299000 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:44.298941    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:45.561891 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:45.561791    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:45.872719 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:45.872624    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:46.145402 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.145360    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"]
Sep 08 20:38:46.149451 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.149419    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.151755 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.151731    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-a6137b-openshift-default-dockercfg-hjjqm\""
Sep 08 20:38:46.164259 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.164180    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"]
Sep 08 20:38:46.172424 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.172392    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:46.216781 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.216747    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn64v\" (UniqueName: \"kubernetes.io/projected/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-kube-api-access-qn64v\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.216781 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.216794    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-workload-certs\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.217048 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.216868    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-envoy\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.217048 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.216913    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-data\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.217048 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.216931    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-workload-socket\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.217048 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.217011    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-podinfo\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.217224 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.217069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istiod-ca-cert\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.217224 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.217110    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-token\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.217224 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.217163    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-credential-socket\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.318534 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.318488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-data\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.318728 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.318553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-workload-socket\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.318728 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.318597    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-podinfo\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.318728 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.318639    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istiod-ca-cert\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.318728 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.318669    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-token\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.318728 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.318707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-credential-socket\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.319021 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.318746    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qn64v\" (UniqueName: \"kubernetes.io/projected/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-kube-api-access-qn64v\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.319021 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.318771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-workload-certs\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.319021 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.318807    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-envoy\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.319021 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.319001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-data\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.319515 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.319219    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-credential-socket\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.319515 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.319464    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-workload-certs\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.319731 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.319702    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istiod-ca-cert\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.319859 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.319831    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-workload-socket\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.321136 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.321115    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-envoy\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.321532 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.321509    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-podinfo\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.329350 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.329324    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-token\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.329448 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.329327    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn64v\" (UniqueName: \"kubernetes.io/projected/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-kube-api-access-qn64v\") pod \"e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") " pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.439584 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.439482    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:46.467504 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.467454    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:46.757194 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.757163    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"]
Sep 08 20:38:46.759393 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:38:46.759364    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f9a1ec4_0aa0_4f5b_95a0_0832ff3b5211.slice/crio-acba6fef45613c2deb6cd9514708ac26570ae8ad2871279f7f85aef884aef236 WatchSource:0}: Error finding container acba6fef45613c2deb6cd9514708ac26570ae8ad2871279f7f85aef884aef236: Status 404 returned error can't find the container with id acba6fef45613c2deb6cd9514708ac26570ae8ad2871279f7f85aef884aef236
Sep 08 20:38:46.761885 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.761837    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:38:46.762022 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.761939    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:38:46.762022 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:46.762014    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:38:47.535577 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:47.535536    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm" event={"ID":"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211","Type":"ContainerStarted","Data":"7d0dfed4c219fa7c4d092a6a2ba933f236843126e736b1b5d4e5fd975d3a0b16"}
Sep 08 20:38:47.535763 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:47.535605    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm" event={"ID":"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211","Type":"ContainerStarted","Data":"acba6fef45613c2deb6cd9514708ac26570ae8ad2871279f7f85aef884aef236"}
Sep 08 20:38:47.560757 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:47.560687    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm" podStartSLOduration=1.560665399 podStartE2EDuration="1.560665399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:38:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:38:47.557422367 +0000 UTC m=+801.104671851" watchObservedRunningTime="2026-09-08 20:38:47.560665399 +0000 UTC m=+801.107914882"
Sep 08 20:38:48.468237 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.468196    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:48.473693 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.473618    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:48.541672 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.541644    2580 generic.go:356] "Generic (PLEG): container finished" podID="c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" containerID="77eaf5ce88dae704fc81e12fc9be79632ac7f53da1cabbb177341cacbc001d9b" exitCode=0
Sep 08 20:38:48.541856 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.541717    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf" event={"ID":"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc","Type":"ContainerDied","Data":"77eaf5ce88dae704fc81e12fc9be79632ac7f53da1cabbb177341cacbc001d9b"}
Sep 08 20:38:48.542168 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.542149    2580 scope.go:117] "RemoveContainer" containerID="77eaf5ce88dae704fc81e12fc9be79632ac7f53da1cabbb177341cacbc001d9b"
Sep 08 20:38:48.543325 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.543306    2580 generic.go:356] "Generic (PLEG): container finished" podID="49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" containerID="2383cf4e77fcf25ed943ede723c62330411140e5bac1370c504e3a4bda9149dd" exitCode=0
Sep 08 20:38:48.543413 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.543323    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g" event={"ID":"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4","Type":"ContainerDied","Data":"2383cf4e77fcf25ed943ede723c62330411140e5bac1370c504e3a4bda9149dd"}
Sep 08 20:38:48.543413 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.543299    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:38:48.543497 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.543438    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:38:48.543497 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.543478    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:38:48.543579 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.543558    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:48.543880 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.543865    2580 scope.go:117] "RemoveContainer" containerID="2383cf4e77fcf25ed943ede723c62330411140e5bac1370c504e3a4bda9149dd"
Sep 08 20:38:48.544479 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.544422    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:38:48.544907 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.544879    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:38:48.545003 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.544951    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:38:48.545054 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.545023    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:38:48.974440 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:48.974337    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:49.077233 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:49.077190    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:49.177939 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:49.177895    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:49.315085 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:49.314989    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:49.444407 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:49.444376    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:49.532502 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:49.532466    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:49.554563 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:49.554526    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf" event={"ID":"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc","Type":"ContainerStarted","Data":"f89679d3e804ba2e37542e2d45e7d4ea281868e0d216acdce83a5ee9cd396ae3"}
Sep 08 20:38:49.566575 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:49.566475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g" event={"ID":"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4","Type":"ContainerStarted","Data":"b3b98f15a06a8cd0620ee1af1f677ac46a3b5f35260c92c4eea3e12f928877a6"}
Sep 08 20:38:50.098325 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.098286    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:50.098490 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.098396    2580 patch_prober.go:28] interesting pod/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 08 20:38:50.098490 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.098443    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf" podUID="c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 08 20:38:50.115053 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.115026    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:50.115345 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.115283    2580 patch_prober.go:28] interesting pod/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 08 20:38:50.115462 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.115446    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g" podUID="49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 08 20:38:50.587541 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.587503    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"]
Sep 08 20:38:50.591253 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.591228    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:38:50.593974 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.593751    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a6137b-serving-cert\""
Sep 08 20:38:50.600464 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.599679    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"]
Sep 08 20:38:50.626019 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.625984    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:50.667731 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.667677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-maas-api-tls\") pod \"maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:38:50.668277 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.668234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-metrics-certs\") pod \"maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:38:50.668771 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.668749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rg6l\" (UniqueName: \"kubernetes.io/projected/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-kube-api-access-7rg6l\") pod \"maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:38:50.669033 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.669014    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:38:50.727489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.727430    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:50.770173 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.770125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-metrics-certs\") pod \"maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:38:50.770394 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.770244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rg6l\" (UniqueName: \"kubernetes.io/projected/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-kube-api-access-7rg6l\") pod \"maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:38:50.770394 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.770284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:38:50.770394 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.770334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-maas-api-tls\") pod \"maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:38:50.770577 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:38:50.770502    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-serving-cert: secret "maas-api-e2e-inf-b-a6137b-serving-cert" not found
Sep 08 20:38:50.770577 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:38:50.770572    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-maas-api-tls podName:1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4 nodeName:}" failed. No retries permitted until 2026-09-08 20:38:51.270549397 +0000 UTC m=+804.817798869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-maas-api-tls") pod "maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6" (UID: "1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4") : secret "maas-api-e2e-inf-b-a6137b-serving-cert" not found
Sep 08 20:38:50.771652 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.771619    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:38:50.775782 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.775755    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-metrics-certs\") pod \"maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:38:50.782268 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.782228    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rg6l\" (UniqueName: \"kubernetes.io/projected/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-kube-api-access-7rg6l\") pod \"maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:38:50.889254 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:50.889213    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:51.098470 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:51.098430    2580 patch_prober.go:28] interesting pod/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 08 20:38:51.098657 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:51.098489    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf" podUID="c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 08 20:38:51.115532 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:51.115502    2580 patch_prober.go:28] interesting pod/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 08 20:38:51.115723 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:51.115704    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g" podUID="49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 08 20:38:51.199040 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:51.198931    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:51.283556 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:51.283512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-maas-api-tls\") pod \"maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:38:51.292327 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:51.292297    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-maas-api-tls\") pod \"maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:38:51.294216 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:51.294183    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:51.509871 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:51.509776    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:38:51.614983 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:51.614465    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:51.832219 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:51.832187    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"]
Sep 08 20:38:51.833860 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:38:51.833832    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e82ddb9_ff2b_41c7_bf4d_1bb1e0e854c4.slice/crio-64136aac637e9ede2274dad97b0615cd68ae785adadcc3e43a029f0473662a14 WatchSource:0}: Error finding container 64136aac637e9ede2274dad97b0615cd68ae785adadcc3e43a029f0473662a14: Status 404 returned error can't find the container with id 64136aac637e9ede2274dad97b0615cd68ae785adadcc3e43a029f0473662a14
Sep 08 20:38:52.098568 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:52.098477    2580 patch_prober.go:28] interesting pod/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 08 20:38:52.098568 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:52.098537    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf" podUID="c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 08 20:38:52.114775 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:52.114744    2580 patch_prober.go:28] interesting pod/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 08 20:38:52.114845 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:52.114796    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g" podUID="49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 08 20:38:52.594158 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:52.594112    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6" event={"ID":"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4","Type":"ContainerStarted","Data":"301be47488830a53635d9a5ea474b07f9231316896c561d45b756c42542317ce"}
Sep 08 20:38:52.594357 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:52.594220    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6" event={"ID":"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4","Type":"ContainerStarted","Data":"64136aac637e9ede2274dad97b0615cd68ae785adadcc3e43a029f0473662a14"}
Sep 08 20:38:52.616818 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:52.616754    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6" podStartSLOduration=2.220165283 podStartE2EDuration="2.616737903s" totalImagesPullingTime="396.57262ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:38:52.615014267 +0000 UTC m=+806.162263752" watchObservedRunningTime="2026-09-08 20:38:52.616737903 +0000 UTC m=+806.163987426"
Sep 08 20:38:52.631798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:52.631764    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:53.098471 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:53.098432    2580 patch_prober.go:28] interesting pod/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 08 20:38:53.098637 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:53.098488    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf" podUID="c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 08 20:38:53.114493 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:53.114461    2580 patch_prober.go:28] interesting pod/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 08 20:38:53.114641 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:53.114514    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g" podUID="49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 08 20:38:53.395485 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:53.395426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:53.599177 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:53.599147    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:38:54.034606 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:54.034558    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:54.098432 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:54.098395    2580 patch_prober.go:28] interesting pod/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 08 20:38:54.098594 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:54.098453    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf" podUID="c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 08 20:38:54.114731 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:54.114701    2580 patch_prober.go:28] interesting pod/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Sep 08 20:38:54.114971 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:54.114936    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g" podUID="49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Sep 08 20:38:54.820866 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:54.820829    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:55.102953 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:55.102871    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:55.103401 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:55.103171    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:55.103914 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:55.103897    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:38:55.306171 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:55.306131    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:55.306535 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:55.306508    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:55.307785 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:55.307765    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:38:55.338632 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:55.338597    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:55.511068 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:55.511033    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:55.612594 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:55.612565    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:38:59.610787 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:38:59.610754    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:39:00.878775 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.878737    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"]
Sep 08 20:39:00.882566 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.882549    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.884062 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.884036    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-model-cache\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.884175 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.884092    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-tmp-dir\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.884175 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.884147    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-dshm\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.884175 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.884165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-home\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.884332 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.884185    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt255\" (UniqueName: \"kubernetes.io/projected/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-kube-api-access-lt255\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.884332 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.884207    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-tls-certs\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.884734 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.884714    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-b7a16698-kserve-self-signed-certs\""
Sep 08 20:39:00.891937 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.891910    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"]
Sep 08 20:39:00.985617 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.985575    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-model-cache\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.985797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.985669    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-tmp-dir\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.985797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.985711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-dshm\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.985912 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.985836    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-home\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.985912 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.985892    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt255\" (UniqueName: \"kubernetes.io/projected/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-kube-api-access-lt255\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.986060 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.985932    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-tls-certs\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.986060 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.986029    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-tmp-dir\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.986474 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.986448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-home\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.986597 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.986476    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-model-cache\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.988006 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.987986    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-dshm\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.988354 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.988333    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-tls-certs\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:00.993556 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:00.993533    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt255\" (UniqueName: \"kubernetes.io/projected/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-kube-api-access-lt255\") pod \"e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") " pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:01.182154 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:01.182055    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:01.192877 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:01.192850    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:01.488280 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:01.488247    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"]
Sep 08 20:39:01.489889 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:39:01.489866    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb958e0b4_05eb_44d3_9d2a_1c8f4b26ff4d.slice/crio-8efdbc351c320e1696625e45f8e3b3311a658cc8be725ed4fc012ad1f998d60a WatchSource:0}: Error finding container 8efdbc351c320e1696625e45f8e3b3311a658cc8be725ed4fc012ad1f998d60a: Status 404 returned error can't find the container with id 8efdbc351c320e1696625e45f8e3b3311a658cc8be725ed4fc012ad1f998d60a
Sep 08 20:39:01.643485 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:01.643439    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2" event={"ID":"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d","Type":"ContainerStarted","Data":"4d4d5efcc70e5d04badeb9894a88c3bd23c0da1129d819af264d9387938f49d5"}
Sep 08 20:39:01.643708 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:01.643498    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2" event={"ID":"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d","Type":"ContainerStarted","Data":"8efdbc351c320e1696625e45f8e3b3311a658cc8be725ed4fc012ad1f998d60a"}
Sep 08 20:39:01.675731 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:01.675673    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2" podStartSLOduration=1.675656918 podStartE2EDuration="1.675656918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:39:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:39:01.67506165 +0000 UTC m=+815.222311133" watchObservedRunningTime="2026-09-08 20:39:01.675656918 +0000 UTC m=+815.222906398"
Sep 08 20:39:08.687918 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:08.687878    2580 generic.go:356] "Generic (PLEG): container finished" podID="1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" containerID="7d0dfed4c219fa7c4d092a6a2ba933f236843126e736b1b5d4e5fd975d3a0b16" exitCode=0
Sep 08 20:39:08.688353 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:08.687951    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm" event={"ID":"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211","Type":"ContainerDied","Data":"7d0dfed4c219fa7c4d092a6a2ba933f236843126e736b1b5d4e5fd975d3a0b16"}
Sep 08 20:39:08.688448 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:08.688434    2580 scope.go:117] "RemoveContainer" containerID="7d0dfed4c219fa7c4d092a6a2ba933f236843126e736b1b5d4e5fd975d3a0b16"
Sep 08 20:39:08.689548 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:08.689520    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:39:08.689612 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:08.689593    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:39:08.689653 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:08.689625    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:39:09.694900 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:09.694861    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm" event={"ID":"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211","Type":"ContainerStarted","Data":"2e48c596c1deaa979a6ee4d14dc407402c5f51d2c34350fe2f40a45f45320928"}
Sep 08 20:39:10.201426 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:10.201387    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:10.467999 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:10.467888    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:39:10.468186 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:10.468084    2580 patch_prober.go:28] interesting pod/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 08 20:39:10.468254 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:10.468172    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm" podUID="1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 08 20:39:10.734431 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:10.734342    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:10.799678 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:10.799647    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:10.877174 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:10.877145    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:10.977114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:10.977080    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:11.039315 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:11.039227    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:11.135652 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:11.135614    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"]
Sep 08 20:39:11.135880 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:11.135850    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf" podUID="c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" containerName="istio-proxy" containerID="cri-o://f89679d3e804ba2e37542e2d45e7d4ea281868e0d216acdce83a5ee9cd396ae3" gracePeriod=30
Sep 08 20:39:11.167734 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:11.167702    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:11.193383 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:11.193333    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:11.193383 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:11.193396    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:11.206024 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:11.205989    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:11.468803 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:11.468756    2580 patch_prober.go:28] interesting pod/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 08 20:39:11.469010 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:11.468824    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm" podUID="1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 08 20:39:11.628575 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:11.628535    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:11.719388 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:11.719300    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:39:11.720311 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:11.720279    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:11.820205 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:11.820167    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:12.134385 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:12.134350    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:12.248355 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:12.248273    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:12.468047 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:12.467920    2580 patch_prober.go:28] interesting pod/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 08 20:39:12.468047 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:12.468032    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm" podUID="1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 08 20:39:13.468423 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:13.468377    2580 patch_prober.go:28] interesting pod/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 08 20:39:13.468831 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:13.468441    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm" podUID="1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 08 20:39:14.406200 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:14.406155    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:14.467901 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:14.467858    2580 patch_prober.go:28] interesting pod/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 08 20:39:14.468097 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:14.467936    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm" podUID="1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 08 20:39:14.686499 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:14.686391    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:14.785705 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:14.785665    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:14.856783 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:14.856744    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:15.083935 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:15.083822    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:15.327025 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:15.326985    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:15.455749 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:15.455717    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:15.471851 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:15.471824    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:39:15.472177 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:15.472153    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:39:15.472816 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:15.472798    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:39:16.396540 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.396473    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:39:16.435898 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.435867    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-data\") pod \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") "
Sep 08 20:39:16.436099 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.435944    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istiod-ca-cert\") pod \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") "
Sep 08 20:39:16.436099 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.436036    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-workload-certs\") pod \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") "
Sep 08 20:39:16.436229 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.436108    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x69bf\" (UniqueName: \"kubernetes.io/projected/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-kube-api-access-x69bf\") pod \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") "
Sep 08 20:39:16.436229 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.436136    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-envoy\") pod \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") "
Sep 08 20:39:16.436229 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.436175    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-workload-socket\") pod \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") "
Sep 08 20:39:16.436229 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.436211    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-token\") pod \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") "
Sep 08 20:39:16.436423 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.436233    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-data" (OuterVolumeSpecName: "istio-data") pod "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" (UID: "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:39:16.436423 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.436265    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-credential-socket\") pod \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") "
Sep 08 20:39:16.436423 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.436292    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-podinfo\") pod \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\" (UID: \"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc\") "
Sep 08 20:39:16.436423 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.436409    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" (UID: "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:39:16.436641 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.436617    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" (UID: "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:39:16.436744 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.436697    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" (UID: "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:39:16.436842 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.436819    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" (UID: "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:39:16.437224 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.437138    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-workload-socket\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:39:16.437224 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.437161    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-credential-socket\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:39:16.437224 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.437176    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-data\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:39:16.437224 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.437188    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istiod-ca-cert\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:39:16.437224 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.437201    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-workload-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:39:16.438855 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.438824    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-kube-api-access-x69bf" (OuterVolumeSpecName: "kube-api-access-x69bf") pod "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" (UID: "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc"). InnerVolumeSpecName "kube-api-access-x69bf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:39:16.438855 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.438838    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-token" (OuterVolumeSpecName: "istio-token") pod "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" (UID: "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:39:16.439028 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.438916    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" (UID: "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:39:16.439028 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.438938    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" (UID: "c2de30cd-cc04-4dec-bfdd-1b2fb23feefc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:39:16.537711 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.537678    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-podinfo\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:39:16.537711 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.537705    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x69bf\" (UniqueName: \"kubernetes.io/projected/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-kube-api-access-x69bf\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:39:16.537711 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.537716    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-envoy\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:39:16.537711 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.537725    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc-istio-token\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:39:16.736271 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.736181    2580 generic.go:356] "Generic (PLEG): container finished" podID="c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" containerID="f89679d3e804ba2e37542e2d45e7d4ea281868e0d216acdce83a5ee9cd396ae3" exitCode=0
Sep 08 20:39:16.736271 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.736230    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"
Sep 08 20:39:16.736448 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.736269    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf" event={"ID":"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc","Type":"ContainerDied","Data":"f89679d3e804ba2e37542e2d45e7d4ea281868e0d216acdce83a5ee9cd396ae3"}
Sep 08 20:39:16.736448 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.736320    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf" event={"ID":"c2de30cd-cc04-4dec-bfdd-1b2fb23feefc","Type":"ContainerDied","Data":"988708c7e8e9d49d0265a51456a09057937583e4a787ba1cd8707447e8a94d76"}
Sep 08 20:39:16.736448 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.736349    2580 scope.go:117] "RemoveContainer" containerID="f89679d3e804ba2e37542e2d45e7d4ea281868e0d216acdce83a5ee9cd396ae3"
Sep 08 20:39:16.746201 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.746182    2580 scope.go:117] "RemoveContainer" containerID="77eaf5ce88dae704fc81e12fc9be79632ac7f53da1cabbb177341cacbc001d9b"
Sep 08 20:39:16.754371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.754352    2580 scope.go:117] "RemoveContainer" containerID="f89679d3e804ba2e37542e2d45e7d4ea281868e0d216acdce83a5ee9cd396ae3"
Sep 08 20:39:16.754882 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:39:16.754831    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f89679d3e804ba2e37542e2d45e7d4ea281868e0d216acdce83a5ee9cd396ae3\": container with ID starting with f89679d3e804ba2e37542e2d45e7d4ea281868e0d216acdce83a5ee9cd396ae3 not found: ID does not exist" containerID="f89679d3e804ba2e37542e2d45e7d4ea281868e0d216acdce83a5ee9cd396ae3"
Sep 08 20:39:16.755006 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.754886    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f89679d3e804ba2e37542e2d45e7d4ea281868e0d216acdce83a5ee9cd396ae3"} err="failed to get container status \"f89679d3e804ba2e37542e2d45e7d4ea281868e0d216acdce83a5ee9cd396ae3\": rpc error: code = NotFound desc = could not find container \"f89679d3e804ba2e37542e2d45e7d4ea281868e0d216acdce83a5ee9cd396ae3\": container with ID starting with f89679d3e804ba2e37542e2d45e7d4ea281868e0d216acdce83a5ee9cd396ae3 not found: ID does not exist"
Sep 08 20:39:16.755006 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.754908    2580 scope.go:117] "RemoveContainer" containerID="77eaf5ce88dae704fc81e12fc9be79632ac7f53da1cabbb177341cacbc001d9b"
Sep 08 20:39:16.755383 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:39:16.755357    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77eaf5ce88dae704fc81e12fc9be79632ac7f53da1cabbb177341cacbc001d9b\": container with ID starting with 77eaf5ce88dae704fc81e12fc9be79632ac7f53da1cabbb177341cacbc001d9b not found: ID does not exist" containerID="77eaf5ce88dae704fc81e12fc9be79632ac7f53da1cabbb177341cacbc001d9b"
Sep 08 20:39:16.755484 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.755390    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77eaf5ce88dae704fc81e12fc9be79632ac7f53da1cabbb177341cacbc001d9b"} err="failed to get container status \"77eaf5ce88dae704fc81e12fc9be79632ac7f53da1cabbb177341cacbc001d9b\": rpc error: code = NotFound desc = could not find container \"77eaf5ce88dae704fc81e12fc9be79632ac7f53da1cabbb177341cacbc001d9b\": container with ID starting with 77eaf5ce88dae704fc81e12fc9be79632ac7f53da1cabbb177341cacbc001d9b not found: ID does not exist"
Sep 08 20:39:16.757164 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.757142    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"]
Sep 08 20:39:16.765556 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.765535    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c73d714b-openshift-default-667cbdd99d-5s5rf"]
Sep 08 20:39:16.924249 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:16.924213    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" path="/var/lib/kubelet/pods/c2de30cd-cc04-4dec-bfdd-1b2fb23feefc/volumes"
Sep 08 20:39:38.811363 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.811326    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"]
Sep 08 20:39:38.811838 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.811748    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" containerName="istio-proxy"
Sep 08 20:39:38.811838 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.811760    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" containerName="istio-proxy"
Sep 08 20:39:38.811838 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.811772    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" containerName="istio-proxy"
Sep 08 20:39:38.811838 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.811778    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" containerName="istio-proxy"
Sep 08 20:39:38.812034 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.811850    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" containerName="istio-proxy"
Sep 08 20:39:38.812034 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.811857    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2de30cd-cc04-4dec-bfdd-1b2fb23feefc" containerName="istio-proxy"
Sep 08 20:39:38.816375 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.816353    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:38.818508 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.818470    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-91d46f\"/\"kube-root-ca.crt\""
Sep 08 20:39:38.819241 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.819218    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-91d46f\"/\"default-dockercfg-ddg8s\""
Sep 08 20:39:38.819355 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.819248    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-91d46f\"/\"openshift-service-ca.crt\""
Sep 08 20:39:38.819355 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.819226    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-91d46f\"/\"autores-model-91d46f-kserve-self-signed-certs\""
Sep 08 20:39:38.824322 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.824293    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"]
Sep 08 20:39:38.844317 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.844288    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:38.950444 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.950408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-model-cache\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:38.950593 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.950460    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9a7b648-0692-42e5-81e4-0c27b7b44793-tls-certs\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:38.950593 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.950547    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-tmp-dir\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:38.950593 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.950585    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwqbr\" (UniqueName: \"kubernetes.io/projected/a9a7b648-0692-42e5-81e4-0c27b7b44793-kube-api-access-nwqbr\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:38.950711 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.950640    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-home\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:38.950711 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:38.950699    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-dshm\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:39.051665 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.051625    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-dshm\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:39.051846 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.051689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-model-cache\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:39.051846 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.051726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9a7b648-0692-42e5-81e4-0c27b7b44793-tls-certs\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:39.051846 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.051762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-tmp-dir\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:39.051846 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.051782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwqbr\" (UniqueName: \"kubernetes.io/projected/a9a7b648-0692-42e5-81e4-0c27b7b44793-kube-api-access-nwqbr\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:39.051846 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.051806    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-home\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:39.052191 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.052164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-model-cache\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:39.052191 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.052180    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-tmp-dir\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:39.052267 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.052227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-home\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:39.053938 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.053910    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-dshm\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:39.054202 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.054183    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9a7b648-0692-42e5-81e4-0c27b7b44793-tls-certs\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:39.059770 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.059753    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwqbr\" (UniqueName: \"kubernetes.io/projected/a9a7b648-0692-42e5-81e4-0c27b7b44793-kube-api-access-nwqbr\") pod \"autores-model-91d46f-kserve-59fd9d9c79-njr7z\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") " pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:39.128805 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.128772    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:39.400753 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.400666    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"]
Sep 08 20:39:39.403943 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:39:39.403917    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9a7b648_0692_42e5_81e4_0c27b7b44793.slice/crio-916553be7bf3374af8151ced81152468361186b5c9af5e58ec51c0deef102e01 WatchSource:0}: Error finding container 916553be7bf3374af8151ced81152468361186b5c9af5e58ec51c0deef102e01: Status 404 returned error can't find the container with id 916553be7bf3374af8151ced81152468361186b5c9af5e58ec51c0deef102e01
Sep 08 20:39:39.860429 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.860347    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z" event={"ID":"a9a7b648-0692-42e5-81e4-0c27b7b44793","Type":"ContainerStarted","Data":"3406846f8fa55ed753188495d8d2e7de27d636424bb444282a148e7c101bc6db"}
Sep 08 20:39:39.860429 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.860388    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z" event={"ID":"a9a7b648-0692-42e5-81e4-0c27b7b44793","Type":"ContainerStarted","Data":"916553be7bf3374af8151ced81152468361186b5c9af5e58ec51c0deef102e01"}
Sep 08 20:39:39.878833 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:39.878783    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z" podStartSLOduration=1.8787678300000001 podStartE2EDuration="1.87876783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:39:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:39:39.876844811 +0000 UTC m=+853.424094291" watchObservedRunningTime="2026-09-08 20:39:39.87876783 +0000 UTC m=+853.426017318"
Sep 08 20:39:46.244316 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:46.244283    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:46.318589 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:46.318543    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:46.383377 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:46.383322    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:46.455568 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:46.455536    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:46.617914 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:46.617832    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:47.007010 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:47.006945    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:47.316374 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:47.316281    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:47.421217 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:47.421186    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:49.129264 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:49.129222    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:49.129671 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:49.129281    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:49.141856 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:49.141826    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:49.804313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:49.804277    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:49.903255 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:49.903214    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:49.922698 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:49.922667    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:39:50.992630 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:50.992591    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:51.091515 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:51.091478    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:51.195152 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:51.195096    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:51.660769 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:51.660736    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:39:51.757993 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:39:51.757949    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:01.998560 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:01.998514    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:02.099341 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:02.099309    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:13.915986 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:13.915926    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:15.762494 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:15.762456    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:21.472457 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:21.472383    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:21.573837 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:21.573762    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:21.666281 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:21.666244    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:21.729824 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:21.729705    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:21.929705 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:21.929676    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:22.767015 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:22.766976    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:23.181629 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:23.181593    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:23.486924 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:23.486806    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:23.580181 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:23.580147    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:23.904669 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:23.904637    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:24.143643 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.143603    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"]
Sep 08 20:40:24.143855 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.143832    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht" podUID="dfa8b41f-4336-4ba5-a6af-bcdf15f75161" containerName="maas-api" containerID="cri-o://9a5a750f626d324e56d6efa1b31bb2f0947b2217ec804e90668eaa271dbb33d0" gracePeriod=30
Sep 08 20:40:24.177296 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.177267    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:24.243635 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.243591    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:24.344169 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.344104    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:24.392882 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.392856    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:40:24.411827 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.411743    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkrbq\" (UniqueName: \"kubernetes.io/projected/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-kube-api-access-mkrbq\") pod \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\" (UID: \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\") "
Sep 08 20:40:24.411827 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.411817    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-metrics-certs\") pod \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\" (UID: \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\") "
Sep 08 20:40:24.412061 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.411988    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-maas-api-tls\") pod \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\" (UID: \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\") "
Sep 08 20:40:24.412061 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.412037    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-openshift-service-ca\") pod \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\" (UID: \"dfa8b41f-4336-4ba5-a6af-bcdf15f75161\") "
Sep 08 20:40:24.412998 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.412731    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dfa8b41f-4336-4ba5-a6af-bcdf15f75161" (UID: "dfa8b41f-4336-4ba5-a6af-bcdf15f75161"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:40:24.415044 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.415015    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-kube-api-access-mkrbq" (OuterVolumeSpecName: "kube-api-access-mkrbq") pod "dfa8b41f-4336-4ba5-a6af-bcdf15f75161" (UID: "dfa8b41f-4336-4ba5-a6af-bcdf15f75161"). InnerVolumeSpecName "kube-api-access-mkrbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:40:24.415292 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.415269    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dfa8b41f-4336-4ba5-a6af-bcdf15f75161" (UID: "dfa8b41f-4336-4ba5-a6af-bcdf15f75161"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:40:24.415450 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.415424    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:24.416545 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.416518    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "dfa8b41f-4336-4ba5-a6af-bcdf15f75161" (UID: "dfa8b41f-4336-4ba5-a6af-bcdf15f75161"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:40:24.513234 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.513183    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-maas-api-tls\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:40:24.513234 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.513227    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-openshift-service-ca\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:40:24.513234 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.513239    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkrbq\" (UniqueName: \"kubernetes.io/projected/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-kube-api-access-mkrbq\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:40:24.513234 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.513248    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfa8b41f-4336-4ba5-a6af-bcdf15f75161-metrics-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:40:24.752279 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:24.752196    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:25.096090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:25.096003    2580 generic.go:356] "Generic (PLEG): container finished" podID="dfa8b41f-4336-4ba5-a6af-bcdf15f75161" containerID="9a5a750f626d324e56d6efa1b31bb2f0947b2217ec804e90668eaa271dbb33d0" exitCode=0
Sep 08 20:40:25.096090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:25.096043    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"
Sep 08 20:40:25.096090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:25.096063    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht" event={"ID":"dfa8b41f-4336-4ba5-a6af-bcdf15f75161","Type":"ContainerDied","Data":"9a5a750f626d324e56d6efa1b31bb2f0947b2217ec804e90668eaa271dbb33d0"}
Sep 08 20:40:25.096555 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:25.096112    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht" event={"ID":"dfa8b41f-4336-4ba5-a6af-bcdf15f75161","Type":"ContainerDied","Data":"d4c40d97b9ba613a4ded1f0192a29ad8ba6b5247528732828b22ce8c0f710ead"}
Sep 08 20:40:25.096555 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:25.096142    2580 scope.go:117] "RemoveContainer" containerID="9a5a750f626d324e56d6efa1b31bb2f0947b2217ec804e90668eaa271dbb33d0"
Sep 08 20:40:25.105750 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:25.105730    2580 scope.go:117] "RemoveContainer" containerID="9a5a750f626d324e56d6efa1b31bb2f0947b2217ec804e90668eaa271dbb33d0"
Sep 08 20:40:25.106033 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:40:25.106009    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a5a750f626d324e56d6efa1b31bb2f0947b2217ec804e90668eaa271dbb33d0\": container with ID starting with 9a5a750f626d324e56d6efa1b31bb2f0947b2217ec804e90668eaa271dbb33d0 not found: ID does not exist" containerID="9a5a750f626d324e56d6efa1b31bb2f0947b2217ec804e90668eaa271dbb33d0"
Sep 08 20:40:25.106131 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:25.106039    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a5a750f626d324e56d6efa1b31bb2f0947b2217ec804e90668eaa271dbb33d0"} err="failed to get container status \"9a5a750f626d324e56d6efa1b31bb2f0947b2217ec804e90668eaa271dbb33d0\": rpc error: code = NotFound desc = could not find container \"9a5a750f626d324e56d6efa1b31bb2f0947b2217ec804e90668eaa271dbb33d0\": container with ID starting with 9a5a750f626d324e56d6efa1b31bb2f0947b2217ec804e90668eaa271dbb33d0 not found: ID does not exist"
Sep 08 20:40:25.115165 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:25.115137    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"]
Sep 08 20:40:25.118778 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:25.118754    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-91d46f-fcd7f54fd-hjhht"]
Sep 08 20:40:26.130366 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:26.130315    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:26.227239 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:26.227195    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:26.924845 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:26.924814    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfa8b41f-4336-4ba5-a6af-bcdf15f75161" path="/var/lib/kubelet/pods/dfa8b41f-4336-4ba5-a6af-bcdf15f75161/volumes"
Sep 08 20:40:27.417341 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:27.417303    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:27.478581 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:27.478546    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:27.650632 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:27.650601    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:27.867861 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:27.867785    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:27.971947 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:27.971909    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:29.726651 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:29.726607    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"]
Sep 08 20:40:29.771458 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:29.771416    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:39.093928 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:39.093894    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:44.746608 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:44.746544    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z" podUID="a9a7b648-0692-42e5-81e4-0c27b7b44793" containerName="main" containerID="cri-o://3406846f8fa55ed753188495d8d2e7de27d636424bb444282a148e7c101bc6db" gracePeriod=45
Sep 08 20:40:44.995882 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:44.995850    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:40:45.117321 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.117231    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-dshm\") pod \"a9a7b648-0692-42e5-81e4-0c27b7b44793\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") "
Sep 08 20:40:45.117321 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.117278    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-home\") pod \"a9a7b648-0692-42e5-81e4-0c27b7b44793\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") "
Sep 08 20:40:45.117321 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.117314    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-model-cache\") pod \"a9a7b648-0692-42e5-81e4-0c27b7b44793\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") "
Sep 08 20:40:45.117608 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.117338    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-tmp-dir\") pod \"a9a7b648-0692-42e5-81e4-0c27b7b44793\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") "
Sep 08 20:40:45.117608 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.117390    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9a7b648-0692-42e5-81e4-0c27b7b44793-tls-certs\") pod \"a9a7b648-0692-42e5-81e4-0c27b7b44793\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") "
Sep 08 20:40:45.117608 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.117444    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwqbr\" (UniqueName: \"kubernetes.io/projected/a9a7b648-0692-42e5-81e4-0c27b7b44793-kube-api-access-nwqbr\") pod \"a9a7b648-0692-42e5-81e4-0c27b7b44793\" (UID: \"a9a7b648-0692-42e5-81e4-0c27b7b44793\") "
Sep 08 20:40:45.117608 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.117592    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a9a7b648-0692-42e5-81e4-0c27b7b44793" (UID: "a9a7b648-0692-42e5-81e4-0c27b7b44793"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:40:45.117775 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.117623    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-home" (OuterVolumeSpecName: "home") pod "a9a7b648-0692-42e5-81e4-0c27b7b44793" (UID: "a9a7b648-0692-42e5-81e4-0c27b7b44793"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:40:45.117775 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.117637    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-model-cache" (OuterVolumeSpecName: "model-cache") pod "a9a7b648-0692-42e5-81e4-0c27b7b44793" (UID: "a9a7b648-0692-42e5-81e4-0c27b7b44793"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:40:45.117893 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.117870    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-home\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:40:45.117893 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.117892    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-model-cache\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:40:45.118044 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.117905    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-tmp-dir\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:40:45.119697 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.119669    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-dshm" (OuterVolumeSpecName: "dshm") pod "a9a7b648-0692-42e5-81e4-0c27b7b44793" (UID: "a9a7b648-0692-42e5-81e4-0c27b7b44793"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:40:45.119815 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.119713    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9a7b648-0692-42e5-81e4-0c27b7b44793-kube-api-access-nwqbr" (OuterVolumeSpecName: "kube-api-access-nwqbr") pod "a9a7b648-0692-42e5-81e4-0c27b7b44793" (UID: "a9a7b648-0692-42e5-81e4-0c27b7b44793"). InnerVolumeSpecName "kube-api-access-nwqbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:40:45.119815 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.119731    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9a7b648-0692-42e5-81e4-0c27b7b44793-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a9a7b648-0692-42e5-81e4-0c27b7b44793" (UID: "a9a7b648-0692-42e5-81e4-0c27b7b44793"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:40:45.205156 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.205118    2580 generic.go:356] "Generic (PLEG): container finished" podID="a9a7b648-0692-42e5-81e4-0c27b7b44793" containerID="3406846f8fa55ed753188495d8d2e7de27d636424bb444282a148e7c101bc6db" exitCode=0
Sep 08 20:40:45.205342 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.205180    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"
Sep 08 20:40:45.205342 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.205211    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z" event={"ID":"a9a7b648-0692-42e5-81e4-0c27b7b44793","Type":"ContainerDied","Data":"3406846f8fa55ed753188495d8d2e7de27d636424bb444282a148e7c101bc6db"}
Sep 08 20:40:45.205342 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.205260    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z" event={"ID":"a9a7b648-0692-42e5-81e4-0c27b7b44793","Type":"ContainerDied","Data":"916553be7bf3374af8151ced81152468361186b5c9af5e58ec51c0deef102e01"}
Sep 08 20:40:45.205342 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.205283    2580 scope.go:117] "RemoveContainer" containerID="3406846f8fa55ed753188495d8d2e7de27d636424bb444282a148e7c101bc6db"
Sep 08 20:40:45.215998 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.215939    2580 scope.go:117] "RemoveContainer" containerID="3406846f8fa55ed753188495d8d2e7de27d636424bb444282a148e7c101bc6db"
Sep 08 20:40:45.216324 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:40:45.216302    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3406846f8fa55ed753188495d8d2e7de27d636424bb444282a148e7c101bc6db\": container with ID starting with 3406846f8fa55ed753188495d8d2e7de27d636424bb444282a148e7c101bc6db not found: ID does not exist" containerID="3406846f8fa55ed753188495d8d2e7de27d636424bb444282a148e7c101bc6db"
Sep 08 20:40:45.216407 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.216331    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3406846f8fa55ed753188495d8d2e7de27d636424bb444282a148e7c101bc6db"} err="failed to get container status \"3406846f8fa55ed753188495d8d2e7de27d636424bb444282a148e7c101bc6db\": rpc error: code = NotFound desc = could not find container \"3406846f8fa55ed753188495d8d2e7de27d636424bb444282a148e7c101bc6db\": container with ID starting with 3406846f8fa55ed753188495d8d2e7de27d636424bb444282a148e7c101bc6db not found: ID does not exist"
Sep 08 20:40:45.218600 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.218577    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9a7b648-0692-42e5-81e4-0c27b7b44793-dshm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:40:45.218745 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.218730    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9a7b648-0692-42e5-81e4-0c27b7b44793-tls-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:40:45.218834 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.218822    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwqbr\" (UniqueName: \"kubernetes.io/projected/a9a7b648-0692-42e5-81e4-0c27b7b44793-kube-api-access-nwqbr\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:40:45.225585 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.225560    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"]
Sep 08 20:40:45.230953 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:45.230930    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-91d46f/autores-model-91d46f-kserve-59fd9d9c79-njr7z"]
Sep 08 20:40:46.924485 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:46.924453    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9a7b648-0692-42e5-81e4-0c27b7b44793" path="/var/lib/kubelet/pods/a9a7b648-0692-42e5-81e4-0c27b7b44793/volumes"
Sep 08 20:40:57.827186 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:57.827151    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:57.897208 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:57.897170    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:57.969457 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:57.969412    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:58.043845 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:58.043813    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:58.157396 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:58.157361    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:58.815896 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:58.815861    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:58.919385 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:58.919354    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:40:59.019575 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:40:59.019527    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:00.293425 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:00.293390    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:05.593445 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.593407    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"]
Sep 08 20:41:05.594031 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.594012    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9a7b648-0692-42e5-81e4-0c27b7b44793" containerName="main"
Sep 08 20:41:05.594129 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.594033    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9a7b648-0692-42e5-81e4-0c27b7b44793" containerName="main"
Sep 08 20:41:05.594129 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.594068    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfa8b41f-4336-4ba5-a6af-bcdf15f75161" containerName="maas-api"
Sep 08 20:41:05.594129 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.594076    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfa8b41f-4336-4ba5-a6af-bcdf15f75161" containerName="maas-api"
Sep 08 20:41:05.594286 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.594160    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfa8b41f-4336-4ba5-a6af-bcdf15f75161" containerName="maas-api"
Sep 08 20:41:05.594286 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.594175    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9a7b648-0692-42e5-81e4-0c27b7b44793" containerName="main"
Sep 08 20:41:05.597233 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.597214    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.600530 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.600508    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6f8575\"/\"default-dockercfg-wv5ds\""
Sep 08 20:41:05.600530 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.600524    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6f8575\"/\"test-model-6f8575-kserve-self-signed-certs\""
Sep 08 20:41:05.600718 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.600541    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6f8575\"/\"kube-root-ca.crt\""
Sep 08 20:41:05.600718 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.600529    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6f8575\"/\"openshift-service-ca.crt\""
Sep 08 20:41:05.605434 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.605410    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"]
Sep 08 20:41:05.615263 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.615235    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:05.624517 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.624492    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-dshm\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.624860 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.624534    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzvvj\" (UniqueName: \"kubernetes.io/projected/b987aeab-4ab2-488d-acfa-2e0704e86a3c-kube-api-access-kzvvj\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.624860 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.624581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-tmp-dir\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.624860 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.624611    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b987aeab-4ab2-488d-acfa-2e0704e86a3c-tls-certs\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.624860 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.624667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-model-cache\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.624860 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.624746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-home\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.725511 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.725478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-dshm\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.725511 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.725515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzvvj\" (UniqueName: \"kubernetes.io/projected/b987aeab-4ab2-488d-acfa-2e0704e86a3c-kube-api-access-kzvvj\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.725719 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.725650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-tmp-dir\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.725719 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.725706    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b987aeab-4ab2-488d-acfa-2e0704e86a3c-tls-certs\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.725817 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.725802    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-model-cache\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.725871 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.725833    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-home\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.726079 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.726056    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-tmp-dir\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.726174 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.726105    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-model-cache\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.726174 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.726168    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-home\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.727793 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.727771    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-dshm\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.728168 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.728149    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b987aeab-4ab2-488d-acfa-2e0704e86a3c-tls-certs\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.733006 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.732984    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzvvj\" (UniqueName: \"kubernetes.io/projected/b987aeab-4ab2-488d-acfa-2e0704e86a3c-kube-api-access-kzvvj\") pod \"test-model-6f8575-kserve-74765d97bc-kt4pl\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") " pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.908216 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.908184    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:05.991108 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:05.991071    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:06.191874 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:06.191850    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"]
Sep 08 20:41:06.194168 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:41:06.194134    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb987aeab_4ab2_488d_acfa_2e0704e86a3c.slice/crio-cdfe2d6ee77a91a9bc11e942349f0ba5ad435cf3b5c6bd35570bc270d6c20c8b WatchSource:0}: Error finding container cdfe2d6ee77a91a9bc11e942349f0ba5ad435cf3b5c6bd35570bc270d6c20c8b: Status 404 returned error can't find the container with id cdfe2d6ee77a91a9bc11e942349f0ba5ad435cf3b5c6bd35570bc270d6c20c8b
Sep 08 20:41:06.310122 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:06.310089    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl" event={"ID":"b987aeab-4ab2-488d-acfa-2e0704e86a3c","Type":"ContainerStarted","Data":"cc2e7ac5db84690d7fffc744143dda3aa166f53d7970cfa32838e579f5b55b80"}
Sep 08 20:41:06.310122 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:06.310128    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl" event={"ID":"b987aeab-4ab2-488d-acfa-2e0704e86a3c","Type":"ContainerStarted","Data":"cdfe2d6ee77a91a9bc11e942349f0ba5ad435cf3b5c6bd35570bc270d6c20c8b"}
Sep 08 20:41:06.329197 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:06.329141    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl" podStartSLOduration=1.329125404 podStartE2EDuration="1.329125404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:41:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:41:06.32757493 +0000 UTC m=+939.874824409" watchObservedRunningTime="2026-09-08 20:41:06.329125404 +0000 UTC m=+939.876374885"
Sep 08 20:41:06.486618 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:06.486518    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:06.591390 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:06.591353    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:06.686306 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:06.686276    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:06.786301 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:06.786196    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:07.160128 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:07.160086    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:07.832184 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:07.832154    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:11.338381 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:11.338346    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:11.966504 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:11.966471    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:13.604317 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:13.604278    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:13.881139 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:13.881095    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"]
Sep 08 20:41:13.881445 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:13.881383    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g" podUID="49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" containerName="istio-proxy" containerID="cri-o://b3b98f15a06a8cd0620ee1af1f677ac46a3b5f35260c92c4eea3e12f928877a6" gracePeriod=30
Sep 08 20:41:15.052528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:15.052490    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:15.908619 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:15.908578    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:15.908805 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:15.908662    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:15.921432 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:15.921404    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:16.372774 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:16.372694    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:41:18.049187 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:18.049151    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:18.148909 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:18.148876    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:19.129565 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.129538    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:41:19.232745 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.232658    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:19.257710 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.257675    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-token\") pod \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") "
Sep 08 20:41:19.257903 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.257726    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-envoy\") pod \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") "
Sep 08 20:41:19.257903 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.257758    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-workload-certs\") pod \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") "
Sep 08 20:41:19.257903 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.257780    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-podinfo\") pod \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") "
Sep 08 20:41:19.257903 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.257817    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-workload-socket\") pod \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") "
Sep 08 20:41:19.257903 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.257860    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6slw\" (UniqueName: \"kubernetes.io/projected/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-kube-api-access-z6slw\") pod \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") "
Sep 08 20:41:19.257903 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.257888    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-credential-socket\") pod \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") "
Sep 08 20:41:19.258257 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.257945    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-data\") pod \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") "
Sep 08 20:41:19.258257 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.258008    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istiod-ca-cert\") pod \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\" (UID: \"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4\") "
Sep 08 20:41:19.258257 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.258122    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" (UID: "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:19.258257 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.258202    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" (UID: "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:19.258456 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.258362    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" (UID: "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:19.258456 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.258391    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-data" (OuterVolumeSpecName: "istio-data") pod "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" (UID: "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:19.258456 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.258436    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" (UID: "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:41:19.258456 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.258440    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-workload-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:41:19.258663 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.258460    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-credential-socket\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:41:19.260114 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.260084    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-token" (OuterVolumeSpecName: "istio-token") pod "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" (UID: "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:41:19.260225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.260125    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" (UID: "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:19.260225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.260151    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" (UID: "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:41:19.260225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.260212    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-kube-api-access-z6slw" (OuterVolumeSpecName: "kube-api-access-z6slw") pod "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" (UID: "49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4"). InnerVolumeSpecName "kube-api-access-z6slw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:41:19.332479 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.332443    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:19.359739 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.359701    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6slw\" (UniqueName: \"kubernetes.io/projected/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-kube-api-access-z6slw\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:41:19.359739 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.359737    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-data\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:41:19.359739 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.359746    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istiod-ca-cert\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:41:19.359939 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.359755    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-token\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:41:19.359939 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.359763    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-envoy\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:41:19.359939 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.359770    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-istio-podinfo\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:41:19.359939 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.359777    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4-workload-socket\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:41:19.379492 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.379451    2580 generic.go:356] "Generic (PLEG): container finished" podID="49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" containerID="b3b98f15a06a8cd0620ee1af1f677ac46a3b5f35260c92c4eea3e12f928877a6" exitCode=0
Sep 08 20:41:19.379704 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.379519    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"
Sep 08 20:41:19.379704 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.379537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g" event={"ID":"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4","Type":"ContainerDied","Data":"b3b98f15a06a8cd0620ee1af1f677ac46a3b5f35260c92c4eea3e12f928877a6"}
Sep 08 20:41:19.379704 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.379607    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g" event={"ID":"49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4","Type":"ContainerDied","Data":"33a4d9652212658650aa7777c7b0257a91456563cf06ee7f1e1b79e43ad5c778"}
Sep 08 20:41:19.379704 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.379639    2580 scope.go:117] "RemoveContainer" containerID="b3b98f15a06a8cd0620ee1af1f677ac46a3b5f35260c92c4eea3e12f928877a6"
Sep 08 20:41:19.389775 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.389755    2580 scope.go:117] "RemoveContainer" containerID="2383cf4e77fcf25ed943ede723c62330411140e5bac1370c504e3a4bda9149dd"
Sep 08 20:41:19.398491 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.398473    2580 scope.go:117] "RemoveContainer" containerID="b3b98f15a06a8cd0620ee1af1f677ac46a3b5f35260c92c4eea3e12f928877a6"
Sep 08 20:41:19.400461 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:41:19.399793    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3b98f15a06a8cd0620ee1af1f677ac46a3b5f35260c92c4eea3e12f928877a6\": container with ID starting with b3b98f15a06a8cd0620ee1af1f677ac46a3b5f35260c92c4eea3e12f928877a6 not found: ID does not exist" containerID="b3b98f15a06a8cd0620ee1af1f677ac46a3b5f35260c92c4eea3e12f928877a6"
Sep 08 20:41:19.400461 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.399836    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3b98f15a06a8cd0620ee1af1f677ac46a3b5f35260c92c4eea3e12f928877a6"} err="failed to get container status \"b3b98f15a06a8cd0620ee1af1f677ac46a3b5f35260c92c4eea3e12f928877a6\": rpc error: code = NotFound desc = could not find container \"b3b98f15a06a8cd0620ee1af1f677ac46a3b5f35260c92c4eea3e12f928877a6\": container with ID starting with b3b98f15a06a8cd0620ee1af1f677ac46a3b5f35260c92c4eea3e12f928877a6 not found: ID does not exist"
Sep 08 20:41:19.400461 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.399862    2580 scope.go:117] "RemoveContainer" containerID="2383cf4e77fcf25ed943ede723c62330411140e5bac1370c504e3a4bda9149dd"
Sep 08 20:41:19.400461 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:41:19.400421    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2383cf4e77fcf25ed943ede723c62330411140e5bac1370c504e3a4bda9149dd\": container with ID starting with 2383cf4e77fcf25ed943ede723c62330411140e5bac1370c504e3a4bda9149dd not found: ID does not exist" containerID="2383cf4e77fcf25ed943ede723c62330411140e5bac1370c504e3a4bda9149dd"
Sep 08 20:41:19.400781 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.400459    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2383cf4e77fcf25ed943ede723c62330411140e5bac1370c504e3a4bda9149dd"} err="failed to get container status \"2383cf4e77fcf25ed943ede723c62330411140e5bac1370c504e3a4bda9149dd\": rpc error: code = NotFound desc = could not find container \"2383cf4e77fcf25ed943ede723c62330411140e5bac1370c504e3a4bda9149dd\": container with ID starting with 2383cf4e77fcf25ed943ede723c62330411140e5bac1370c504e3a4bda9149dd not found: ID does not exist"
Sep 08 20:41:19.401649 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.401627    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:19.402488 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.402467    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"]
Sep 08 20:41:19.406177 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.406152    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-91d46f-openshift-default-7f4dcf76bb-9tn9g"]
Sep 08 20:41:19.838263 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.838222    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:19.958401 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:19.958364    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:20.924235 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:20.924197    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" path="/var/lib/kubelet/pods/49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4/volumes"
Sep 08 20:41:33.083583 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:33.083541    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:34.705381 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:34.705343    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"]
Sep 08 20:41:34.729420 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:34.729384    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:39.176753 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:39.176718    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:39.275747 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:39.275714    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:39.421002 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:39.420970    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:42.975205 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:42.975167    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:48.398363 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.398328    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"]
Sep 08 20:41:48.398775 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.398735    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" containerName="istio-proxy"
Sep 08 20:41:48.398775 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.398747    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" containerName="istio-proxy"
Sep 08 20:41:48.398849 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.398825    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" containerName="istio-proxy"
Sep 08 20:41:48.398849 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.398834    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" containerName="istio-proxy"
Sep 08 20:41:48.403131 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.403109    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.405402 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.405378    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a6137b\"/\"openshift-service-ca.crt\""
Sep 08 20:41:48.406082 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.406063    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a6137b\"/\"test-model-a6137b-kserve-self-signed-certs\""
Sep 08 20:41:48.406198 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.406063    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a6137b\"/\"kube-root-ca.crt\""
Sep 08 20:41:48.406266 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.406228    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a6137b\"/\"default-dockercfg-57k46\""
Sep 08 20:41:48.414650 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.414623    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"]
Sep 08 20:41:48.422431 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.422401    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:48.436572 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.436531    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-tmp-dir\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.436755 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.436589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-dshm\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.436755 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.436656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-home\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.436755 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.436724    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-model-cache\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.436897 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.436775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8hfz\" (UniqueName: \"kubernetes.io/projected/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-kube-api-access-v8hfz\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.436897 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.436807    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-tls-certs\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.537719 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.537680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-model-cache\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.537719 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.537724    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8hfz\" (UniqueName: \"kubernetes.io/projected/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-kube-api-access-v8hfz\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.538033 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.537745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-tls-certs\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.538033 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.537858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-tmp-dir\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.538033 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.537915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-dshm\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.538033 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.538011    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-home\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.538247 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.538196    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-tmp-dir\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.538429 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.538387    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-home\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.538556 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.538514    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-model-cache\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.540337 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.540311    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-dshm\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.540870 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.540842    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-tls-certs\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.546703 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.546676    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8hfz\" (UniqueName: \"kubernetes.io/projected/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-kube-api-access-v8hfz\") pod \"test-model-a6137b-kserve-6fb68876cc-bg67w\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") " pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.712823 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.712726    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:48.987878 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:48.987850    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"]
Sep 08 20:41:48.990382 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:41:48.990356    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ff2c5bd_5eb0_4312_93a2_06ad2231d55b.slice/crio-6b56108aeb06a4b1578b59de247a4f901ce6f5e5b0ea8a897a26e0aeab26473d WatchSource:0}: Error finding container 6b56108aeb06a4b1578b59de247a4f901ce6f5e5b0ea8a897a26e0aeab26473d: Status 404 returned error can't find the container with id 6b56108aeb06a4b1578b59de247a4f901ce6f5e5b0ea8a897a26e0aeab26473d
Sep 08 20:41:49.534321 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:49.534283    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w" event={"ID":"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b","Type":"ContainerStarted","Data":"cde95f77ba593213aedcb50672cc88a53cdd8e6969e690724aa8a2f1f81dc031"}
Sep 08 20:41:49.534321 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:49.534326    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w" event={"ID":"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b","Type":"ContainerStarted","Data":"6b56108aeb06a4b1578b59de247a4f901ce6f5e5b0ea8a897a26e0aeab26473d"}
Sep 08 20:41:49.552347 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:49.552288    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w" podStartSLOduration=1.552270531 podStartE2EDuration="1.552270531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:41:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:41:49.550235136 +0000 UTC m=+983.097484620" watchObservedRunningTime="2026-09-08 20:41:49.552270531 +0000 UTC m=+983.099520048"
Sep 08 20:41:49.726430 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:49.726389    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2" podUID="b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d" containerName="main" containerID="cri-o://4d4d5efcc70e5d04badeb9894a88c3bd23c0da1129d819af264d9387938f49d5" gracePeriod=45
Sep 08 20:41:49.981017 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:49.980989    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:41:50.049733 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.049695    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lt255\" (UniqueName: \"kubernetes.io/projected/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-kube-api-access-lt255\") pod \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") "
Sep 08 20:41:50.049924 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.049762    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-tmp-dir\") pod \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") "
Sep 08 20:41:50.049924 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.049785    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-home\") pod \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") "
Sep 08 20:41:50.049924 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.049804    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-model-cache\") pod \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") "
Sep 08 20:41:50.049924 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.049848    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-tls-certs\") pod \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") "
Sep 08 20:41:50.050180 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.049934    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-dshm\") pod \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\" (UID: \"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d\") "
Sep 08 20:41:50.050180 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.050119    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-model-cache" (OuterVolumeSpecName: "model-cache") pod "b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d" (UID: "b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:50.050180 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.050143    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d" (UID: "b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:50.050329 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.050172    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-home" (OuterVolumeSpecName: "home") pod "b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d" (UID: "b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:50.050329 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.050277    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-tmp-dir\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:41:50.050329 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.050297    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-home\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:41:50.050329 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.050311    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-model-cache\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:41:50.052188 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.052160    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d" (UID: "b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:41:50.052188 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.052159    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-dshm" (OuterVolumeSpecName: "dshm") pod "b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d" (UID: "b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:41:50.052326 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.052198    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-kube-api-access-lt255" (OuterVolumeSpecName: "kube-api-access-lt255") pod "b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d" (UID: "b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d"). InnerVolumeSpecName "kube-api-access-lt255". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:41:50.150802 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.150764    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-dshm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:41:50.150802 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.150799    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt255\" (UniqueName: \"kubernetes.io/projected/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-kube-api-access-lt255\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:41:50.151026 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.150815    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d-tls-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:41:50.540127 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.540039    2580 generic.go:356] "Generic (PLEG): container finished" podID="b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d" containerID="4d4d5efcc70e5d04badeb9894a88c3bd23c0da1129d819af264d9387938f49d5" exitCode=0
Sep 08 20:41:50.540127 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.540084    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"
Sep 08 20:41:50.540562 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.540129    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2" event={"ID":"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d","Type":"ContainerDied","Data":"4d4d5efcc70e5d04badeb9894a88c3bd23c0da1129d819af264d9387938f49d5"}
Sep 08 20:41:50.540562 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.540188    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2" event={"ID":"b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d","Type":"ContainerDied","Data":"8efdbc351c320e1696625e45f8e3b3311a658cc8be725ed4fc012ad1f998d60a"}
Sep 08 20:41:50.540562 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.540212    2580 scope.go:117] "RemoveContainer" containerID="4d4d5efcc70e5d04badeb9894a88c3bd23c0da1129d819af264d9387938f49d5"
Sep 08 20:41:50.552090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.552066    2580 scope.go:117] "RemoveContainer" containerID="4d4d5efcc70e5d04badeb9894a88c3bd23c0da1129d819af264d9387938f49d5"
Sep 08 20:41:50.552375 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:41:50.552351    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d4d5efcc70e5d04badeb9894a88c3bd23c0da1129d819af264d9387938f49d5\": container with ID starting with 4d4d5efcc70e5d04badeb9894a88c3bd23c0da1129d819af264d9387938f49d5 not found: ID does not exist" containerID="4d4d5efcc70e5d04badeb9894a88c3bd23c0da1129d819af264d9387938f49d5"
Sep 08 20:41:50.552455 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.552389    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d4d5efcc70e5d04badeb9894a88c3bd23c0da1129d819af264d9387938f49d5"} err="failed to get container status \"4d4d5efcc70e5d04badeb9894a88c3bd23c0da1129d819af264d9387938f49d5\": rpc error: code = NotFound desc = could not find container \"4d4d5efcc70e5d04badeb9894a88c3bd23c0da1129d819af264d9387938f49d5\": container with ID starting with 4d4d5efcc70e5d04badeb9894a88c3bd23c0da1129d819af264d9387938f49d5 not found: ID does not exist"
Sep 08 20:41:50.561172 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.561146    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"]
Sep 08 20:41:50.566092 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.566063    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-b7a16698-kserve-8687fb58fc-hxgl2"]
Sep 08 20:41:50.924629 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:50.924593    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d" path="/var/lib/kubelet/pods/b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d/volumes"
Sep 08 20:41:54.204384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.204348    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"]
Sep 08 20:41:54.204747 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.204740    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" containerName="istio-proxy"
Sep 08 20:41:54.204794 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.204752    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="49dbefb7-1b13-4c42-9a1e-dfdf5b5461a4" containerName="istio-proxy"
Sep 08 20:41:54.204794 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.204775    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d" containerName="main"
Sep 08 20:41:54.204794 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.204780    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d" containerName="main"
Sep 08 20:41:54.204887 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.204845    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b958e0b4-05eb-44d3-9d2a-1c8f4b26ff4d" containerName="main"
Sep 08 20:41:54.207798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.207778    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.211330 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.211307    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-b433c5\"/\"openshift-service-ca.crt\""
Sep 08 20:41:54.211507 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.211489    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-b433c5\"/\"kube-root-ca.crt\""
Sep 08 20:41:54.211887 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.211599    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-b433c5\"/\"explicit-model-b433c5-kserve-self-signed-certs\""
Sep 08 20:41:54.212286 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.212128    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-b433c5\"/\"default-dockercfg-l929z\""
Sep 08 20:41:54.217554 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.217530    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"]
Sep 08 20:41:54.228195 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.228166    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:54.295531 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.295498    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-home\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.295721 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.295550    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-tls-certs\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.295721 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.295631    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-dshm\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.295721 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.295671    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-tmp-dir\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.295721 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.295693    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrkhs\" (UniqueName: \"kubernetes.io/projected/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-kube-api-access-hrkhs\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.295861 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.295773    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-model-cache\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.396556 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.396521    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrkhs\" (UniqueName: \"kubernetes.io/projected/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-kube-api-access-hrkhs\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.396729 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.396614    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-model-cache\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.396729 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.396658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-home\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.396729 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.396702    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-tls-certs\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.396885 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.396748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-dshm\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.396973 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.396930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-tmp-dir\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.397124 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.397097    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-model-cache\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.397197 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.397125    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-home\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.397343 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.397207    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-tmp-dir\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.399186 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.399161    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-dshm\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.399409 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.399392    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-tls-certs\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.406567 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.406537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrkhs\" (UniqueName: \"kubernetes.io/projected/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-kube-api-access-hrkhs\") pod \"explicit-model-b433c5-kserve-5754559765-xcqh7\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") " pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.523925 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.523829    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:41:54.807093 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:54.807057    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"]
Sep 08 20:41:54.807560 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:41:54.807534    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69c758cd_d9e3_49b8_9c86_75e9d2e376e1.slice/crio-a25ff35e567b4c7af8ec67d7cfefaf8b063c51c5ff61aa46ca56f8538a84db89 WatchSource:0}: Error finding container a25ff35e567b4c7af8ec67d7cfefaf8b063c51c5ff61aa46ca56f8538a84db89: Status 404 returned error can't find the container with id a25ff35e567b4c7af8ec67d7cfefaf8b063c51c5ff61aa46ca56f8538a84db89
Sep 08 20:41:55.257650 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:55.257620    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:41:55.567211 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:55.567116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7" event={"ID":"69c758cd-d9e3-49b8-9c86-75e9d2e376e1","Type":"ContainerStarted","Data":"d4fb00050b4cc1fb0c2df99304d92fbad75434e43f71938d15a2ace96d6bb6db"}
Sep 08 20:41:55.567211 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:55.567158    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7" event={"ID":"69c758cd-d9e3-49b8-9c86-75e9d2e376e1","Type":"ContainerStarted","Data":"a25ff35e567b4c7af8ec67d7cfefaf8b063c51c5ff61aa46ca56f8538a84db89"}
Sep 08 20:41:55.583182 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:55.583113    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7" podStartSLOduration=1.583093788 podStartE2EDuration="1.583093788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:41:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:41:55.582766554 +0000 UTC m=+989.130016036" watchObservedRunningTime="2026-09-08 20:41:55.583093788 +0000 UTC m=+989.130343270"
Sep 08 20:41:58.713879 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:58.713832    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:58.713879 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:58.713888    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:58.726643 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:58.726612    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:41:59.598090 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:41:59.598058    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:42:01.204870 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:01.204835    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:01.303802 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:01.303769    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:01.404649 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:01.404607    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:04.524175 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:04.524128    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:42:04.524701 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:04.524228    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:42:04.537380 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:04.537340    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:42:04.624021 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:04.623979    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:42:08.070384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:08.070348    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"]
Sep 08 20:42:08.091060 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:08.091031    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:11.815357 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:11.815319    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:12.571844 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:12.571809    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:18.309616 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:18.309580    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:18.410225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:18.410190    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:18.509228 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:18.509178    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:18.606438 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:18.606345    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:18.748701 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:18.748661    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:18.851938 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:18.851905    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:18.949439 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:18.949404    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:19.049212 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:19.049172    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:19.246788 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:19.246696    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:19.347588 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:19.347537    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:19.449192 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:19.449151    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:20.877137 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:20.877101    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:23.091364 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.091323    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7" podUID="69c758cd-d9e3-49b8-9c86-75e9d2e376e1" containerName="main" containerID="cri-o://d4fb00050b4cc1fb0c2df99304d92fbad75434e43f71938d15a2ace96d6bb6db" gracePeriod=45
Sep 08 20:42:23.348532 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.348469    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:42:23.393654 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.393615    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrkhs\" (UniqueName: \"kubernetes.io/projected/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-kube-api-access-hrkhs\") pod \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") "
Sep 08 20:42:23.393824 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.393667    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-tmp-dir\") pod \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") "
Sep 08 20:42:23.393824 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.393722    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-home\") pod \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") "
Sep 08 20:42:23.393905 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.393834    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-tls-certs\") pod \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") "
Sep 08 20:42:23.393905 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.393874    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-dshm\") pod \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") "
Sep 08 20:42:23.393905 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.393896    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-model-cache\") pod \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\" (UID: \"69c758cd-d9e3-49b8-9c86-75e9d2e376e1\") "
Sep 08 20:42:23.394092 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.393950    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "69c758cd-d9e3-49b8-9c86-75e9d2e376e1" (UID: "69c758cd-d9e3-49b8-9c86-75e9d2e376e1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:42:23.394151 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.394079    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-home" (OuterVolumeSpecName: "home") pod "69c758cd-d9e3-49b8-9c86-75e9d2e376e1" (UID: "69c758cd-d9e3-49b8-9c86-75e9d2e376e1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:42:23.394299 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.394269    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-model-cache" (OuterVolumeSpecName: "model-cache") pod "69c758cd-d9e3-49b8-9c86-75e9d2e376e1" (UID: "69c758cd-d9e3-49b8-9c86-75e9d2e376e1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:42:23.394364 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.394319    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-home\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:42:23.394409 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.394365    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-tmp-dir\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:42:23.396118 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.396096    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "69c758cd-d9e3-49b8-9c86-75e9d2e376e1" (UID: "69c758cd-d9e3-49b8-9c86-75e9d2e376e1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:42:23.396340 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.396311    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-dshm" (OuterVolumeSpecName: "dshm") pod "69c758cd-d9e3-49b8-9c86-75e9d2e376e1" (UID: "69c758cd-d9e3-49b8-9c86-75e9d2e376e1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:42:23.396386 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.396322    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-kube-api-access-hrkhs" (OuterVolumeSpecName: "kube-api-access-hrkhs") pod "69c758cd-d9e3-49b8-9c86-75e9d2e376e1" (UID: "69c758cd-d9e3-49b8-9c86-75e9d2e376e1"). InnerVolumeSpecName "kube-api-access-hrkhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:42:23.495204 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.495151    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-tls-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:42:23.495204 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.495194    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-dshm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:42:23.495204 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.495204    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-model-cache\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:42:23.495204 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.495213    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrkhs\" (UniqueName: \"kubernetes.io/projected/69c758cd-d9e3-49b8-9c86-75e9d2e376e1-kube-api-access-hrkhs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:42:23.709532 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.709485    2580 generic.go:356] "Generic (PLEG): container finished" podID="69c758cd-d9e3-49b8-9c86-75e9d2e376e1" containerID="d4fb00050b4cc1fb0c2df99304d92fbad75434e43f71938d15a2ace96d6bb6db" exitCode=0
Sep 08 20:42:23.709532 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.709538    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"
Sep 08 20:42:23.709802 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.709575    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7" event={"ID":"69c758cd-d9e3-49b8-9c86-75e9d2e376e1","Type":"ContainerDied","Data":"d4fb00050b4cc1fb0c2df99304d92fbad75434e43f71938d15a2ace96d6bb6db"}
Sep 08 20:42:23.709802 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.709621    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7" event={"ID":"69c758cd-d9e3-49b8-9c86-75e9d2e376e1","Type":"ContainerDied","Data":"a25ff35e567b4c7af8ec67d7cfefaf8b063c51c5ff61aa46ca56f8538a84db89"}
Sep 08 20:42:23.709802 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.709639    2580 scope.go:117] "RemoveContainer" containerID="d4fb00050b4cc1fb0c2df99304d92fbad75434e43f71938d15a2ace96d6bb6db"
Sep 08 20:42:23.718922 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.718896    2580 scope.go:117] "RemoveContainer" containerID="d4fb00050b4cc1fb0c2df99304d92fbad75434e43f71938d15a2ace96d6bb6db"
Sep 08 20:42:23.719224 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:42:23.719202    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4fb00050b4cc1fb0c2df99304d92fbad75434e43f71938d15a2ace96d6bb6db\": container with ID starting with d4fb00050b4cc1fb0c2df99304d92fbad75434e43f71938d15a2ace96d6bb6db not found: ID does not exist" containerID="d4fb00050b4cc1fb0c2df99304d92fbad75434e43f71938d15a2ace96d6bb6db"
Sep 08 20:42:23.719277 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.719232    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4fb00050b4cc1fb0c2df99304d92fbad75434e43f71938d15a2ace96d6bb6db"} err="failed to get container status \"d4fb00050b4cc1fb0c2df99304d92fbad75434e43f71938d15a2ace96d6bb6db\": rpc error: code = NotFound desc = could not find container \"d4fb00050b4cc1fb0c2df99304d92fbad75434e43f71938d15a2ace96d6bb6db\": container with ID starting with d4fb00050b4cc1fb0c2df99304d92fbad75434e43f71938d15a2ace96d6bb6db not found: ID does not exist"
Sep 08 20:42:23.730590 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.730559    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"]
Sep 08 20:42:23.734104 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:23.734083    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-b433c5/explicit-model-b433c5-kserve-5754559765-xcqh7"]
Sep 08 20:42:24.924361 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:24.924330    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69c758cd-d9e3-49b8-9c86-75e9d2e376e1" path="/var/lib/kubelet/pods/69c758cd-d9e3-49b8-9c86-75e9d2e376e1/volumes"
Sep 08 20:42:25.495840 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:25.495804    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:25.555285 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:25.555250    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:25.758491 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:25.758386    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:26.191748 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:26.191718    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:26.495619 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:26.495514    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:26.633193 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:26.633150    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:26.986759 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:26.986720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:27.050427 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:27.050386    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:27.629199 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:27.629162    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:28.072555 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:28.072453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:28.164577 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:28.164532    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:28.266770 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:28.266729    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:33.545391 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:33.545353    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:33.645732 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:33.645698    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:33.764952 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:33.764905    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:34.064375 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:34.064323    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:34.321483 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:34.321403    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:36.851269 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:36.851234    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:37.341610 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:37.341571    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:38.154264 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:38.154227    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:38.255796 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:38.255768    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:38.482688 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:38.482596    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:38.602560 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:38.602527    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:38.708726 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:38.708688    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:59.312991 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.312933    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"]
Sep 08 20:42:59.313524 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.313507    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69c758cd-d9e3-49b8-9c86-75e9d2e376e1" containerName="main"
Sep 08 20:42:59.313524 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.313525    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="69c758cd-d9e3-49b8-9c86-75e9d2e376e1" containerName="main"
Sep 08 20:42:59.313702 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.313642    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="69c758cd-d9e3-49b8-9c86-75e9d2e376e1" containerName="main"
Sep 08 20:42:59.317073 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.317051    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.319345 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.319322    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-458e34\"/\"openshift-service-ca.crt\""
Sep 08 20:42:59.320279 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.320242    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-458e34\"/\"kube-root-ca.crt\""
Sep 08 20:42:59.320553 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.320409    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-458e34\"/\"noten-model-458e34-kserve-self-signed-certs\""
Sep 08 20:42:59.320553 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.320379    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-458e34\"/\"default-dockercfg-q7jhd\""
Sep 08 20:42:59.323054 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.323028    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"]
Sep 08 20:42:59.342394 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.342364    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:42:59.347522 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.347499    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2p89\" (UniqueName: \"kubernetes.io/projected/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-kube-api-access-c2p89\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.347641 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.347530    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-tls-certs\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.347704 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.347650    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-tmp-dir\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.347759 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.347714    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-dshm\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.347806 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.347754    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-home\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.347844 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.347831    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-model-cache\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.449015 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.448975    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-tmp-dir\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.449194 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.449040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-dshm\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.449194 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.449068    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-home\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.449194 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.449106    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-model-cache\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.449194 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.449133    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2p89\" (UniqueName: \"kubernetes.io/projected/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-kube-api-access-c2p89\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.449194 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.449159    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-tls-certs\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.449402 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.449373    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-tmp-dir\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.449402 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.449389    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-home\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.449646 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.449627    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-model-cache\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.451353 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.451328    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-dshm\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.451634 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.451614    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-tls-certs\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.456791 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.456764    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2p89\" (UniqueName: \"kubernetes.io/projected/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-kube-api-access-c2p89\") pod \"noten-model-458e34-kserve-7dc58859ff-tsh4n\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") " pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.627783 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.627750    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:42:59.901881 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:42:59.901854    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"]
Sep 08 20:42:59.903823 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:42:59.903785    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c40aad2_190c_4d0b_8e20_df0ef325fcdd.slice/crio-2ebb09ca79785c9de810f0df73516cf50982cbb1440b193fa2e92842fe2d0a7d WatchSource:0}: Error finding container 2ebb09ca79785c9de810f0df73516cf50982cbb1440b193fa2e92842fe2d0a7d: Status 404 returned error can't find the container with id 2ebb09ca79785c9de810f0df73516cf50982cbb1440b193fa2e92842fe2d0a7d
Sep 08 20:43:00.894660 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:00.894621    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n" event={"ID":"8c40aad2-190c-4d0b-8e20-df0ef325fcdd","Type":"ContainerStarted","Data":"382803a0d4daafa981d1fd66f9c957dfcf2b0c0a82e8c7dd52ee99f4f7c467b6"}
Sep 08 20:43:00.895092 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:00.894668    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n" event={"ID":"8c40aad2-190c-4d0b-8e20-df0ef325fcdd","Type":"ContainerStarted","Data":"2ebb09ca79785c9de810f0df73516cf50982cbb1440b193fa2e92842fe2d0a7d"}
Sep 08 20:43:00.912951 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:00.912889    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n" podStartSLOduration=1.9128719090000001 podStartE2EDuration="1.912871909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:42:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:43:00.910128474 +0000 UTC m=+1054.457377954" watchObservedRunningTime="2026-09-08 20:43:00.912871909 +0000 UTC m=+1054.460121391"
Sep 08 20:43:01.758443 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:01.758410    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:02.396496 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:02.396468    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:02.491352 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:02.491317    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:02.810947 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:02.810862    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:03.575737 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:03.575695    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"]
Sep 08 20:43:03.586017 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:03.585988    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:05.178939 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:05.178895    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"]
Sep 08 20:43:05.199031 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:05.199003    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:05.312494 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:05.312453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:05.525654 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:05.525544    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:05.636184 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:05.636154    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:06.178235 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:06.178200    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"]
Sep 08 20:43:06.188174 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:06.188144    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:06.426918 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:06.426874    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:08.529512 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:08.529483    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:08.594381 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:08.594346    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:09.142493 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:09.142459    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:09.628211 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:09.628175    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:43:09.671201 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:09.671162    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:09.731104 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:09.731067    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:10.646892 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:10.646858    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:16.499068 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:16.499038    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:17.028547 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.028512    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"]
Sep 08 20:43:17.028775 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.028730    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl" podUID="6b705417-23b4-4c43-a46d-cf5c59b3fc52" containerName="maas-api" containerID="cri-o://9c5a0acbf7ab2bc0f7b0008cac6698441e1de8246062272e4b72d8069ee51bed" gracePeriod=30
Sep 08 20:43:17.065105 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.065075    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:17.147792 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.147758    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:17.223430 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.223396    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:17.277745 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.277722    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:43:17.288477 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.288403    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:17.428779 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.428748    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b705417-23b4-4c43-a46d-cf5c59b3fc52-maas-api-tls\") pod \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\" (UID: \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\") "
Sep 08 20:43:17.429008 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.428797    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b705417-23b4-4c43-a46d-cf5c59b3fc52-metrics-certs\") pod \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\" (UID: \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\") "
Sep 08 20:43:17.429008 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.428852    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p89fd\" (UniqueName: \"kubernetes.io/projected/6b705417-23b4-4c43-a46d-cf5c59b3fc52-kube-api-access-p89fd\") pod \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\" (UID: \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\") "
Sep 08 20:43:17.429008 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.428940    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b705417-23b4-4c43-a46d-cf5c59b3fc52-openshift-service-ca\") pod \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\" (UID: \"6b705417-23b4-4c43-a46d-cf5c59b3fc52\") "
Sep 08 20:43:17.429302 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.429280    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b705417-23b4-4c43-a46d-cf5c59b3fc52-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6b705417-23b4-4c43-a46d-cf5c59b3fc52" (UID: "6b705417-23b4-4c43-a46d-cf5c59b3fc52"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:43:17.429399 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.429386    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b705417-23b4-4c43-a46d-cf5c59b3fc52-openshift-service-ca\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:17.430991 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.430966    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b705417-23b4-4c43-a46d-cf5c59b3fc52-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6b705417-23b4-4c43-a46d-cf5c59b3fc52" (UID: "6b705417-23b4-4c43-a46d-cf5c59b3fc52"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:43:17.431417 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.431394    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b705417-23b4-4c43-a46d-cf5c59b3fc52-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6b705417-23b4-4c43-a46d-cf5c59b3fc52" (UID: "6b705417-23b4-4c43-a46d-cf5c59b3fc52"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:43:17.431505 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.431425    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b705417-23b4-4c43-a46d-cf5c59b3fc52-kube-api-access-p89fd" (OuterVolumeSpecName: "kube-api-access-p89fd") pod "6b705417-23b4-4c43-a46d-cf5c59b3fc52" (UID: "6b705417-23b4-4c43-a46d-cf5c59b3fc52"). InnerVolumeSpecName "kube-api-access-p89fd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:43:17.530025 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.529986    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p89fd\" (UniqueName: \"kubernetes.io/projected/6b705417-23b4-4c43-a46d-cf5c59b3fc52-kube-api-access-p89fd\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:17.530025 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.530016    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b705417-23b4-4c43-a46d-cf5c59b3fc52-maas-api-tls\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:17.530025 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.530026    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b705417-23b4-4c43-a46d-cf5c59b3fc52-metrics-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:17.629183 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.629141    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:17.988433 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.988348    2580 generic.go:356] "Generic (PLEG): container finished" podID="6b705417-23b4-4c43-a46d-cf5c59b3fc52" containerID="9c5a0acbf7ab2bc0f7b0008cac6698441e1de8246062272e4b72d8069ee51bed" exitCode=0
Sep 08 20:43:17.988433 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.988393    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"
Sep 08 20:43:17.988433 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.988400    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl" event={"ID":"6b705417-23b4-4c43-a46d-cf5c59b3fc52","Type":"ContainerDied","Data":"9c5a0acbf7ab2bc0f7b0008cac6698441e1de8246062272e4b72d8069ee51bed"}
Sep 08 20:43:17.988655 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.988441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl" event={"ID":"6b705417-23b4-4c43-a46d-cf5c59b3fc52","Type":"ContainerDied","Data":"25aab8421de6d8d182252dc5dfc04c19a4fe5ba905e82de2312454c9771adfa5"}
Sep 08 20:43:17.988655 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.988465    2580 scope.go:117] "RemoveContainer" containerID="9c5a0acbf7ab2bc0f7b0008cac6698441e1de8246062272e4b72d8069ee51bed"
Sep 08 20:43:17.997905 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.997880    2580 scope.go:117] "RemoveContainer" containerID="9c5a0acbf7ab2bc0f7b0008cac6698441e1de8246062272e4b72d8069ee51bed"
Sep 08 20:43:17.998239 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:43:17.998216    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c5a0acbf7ab2bc0f7b0008cac6698441e1de8246062272e4b72d8069ee51bed\": container with ID starting with 9c5a0acbf7ab2bc0f7b0008cac6698441e1de8246062272e4b72d8069ee51bed not found: ID does not exist" containerID="9c5a0acbf7ab2bc0f7b0008cac6698441e1de8246062272e4b72d8069ee51bed"
Sep 08 20:43:17.998313 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:17.998245    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c5a0acbf7ab2bc0f7b0008cac6698441e1de8246062272e4b72d8069ee51bed"} err="failed to get container status \"9c5a0acbf7ab2bc0f7b0008cac6698441e1de8246062272e4b72d8069ee51bed\": rpc error: code = NotFound desc = could not find container \"9c5a0acbf7ab2bc0f7b0008cac6698441e1de8246062272e4b72d8069ee51bed\": container with ID starting with 9c5a0acbf7ab2bc0f7b0008cac6698441e1de8246062272e4b72d8069ee51bed not found: ID does not exist"
Sep 08 20:43:18.010543 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.010510    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"]
Sep 08 20:43:18.013882 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.013858    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6f8575-5d55458675-tswtl"]
Sep 08 20:43:18.596046 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.596002    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl" podUID="b987aeab-4ab2-488d-acfa-2e0704e86a3c" containerName="main" containerID="cri-o://cc2e7ac5db84690d7fffc744143dda3aa166f53d7970cfa32838e579f5b55b80" gracePeriod=45
Sep 08 20:43:18.845393 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.845368    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:43:18.924981 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.924918    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b705417-23b4-4c43-a46d-cf5c59b3fc52" path="/var/lib/kubelet/pods/6b705417-23b4-4c43-a46d-cf5c59b3fc52/volumes"
Sep 08 20:43:18.943076 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.943044    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-home\") pod \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") "
Sep 08 20:43:18.943229 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.943142    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-dshm\") pod \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") "
Sep 08 20:43:18.943229 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.943180    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-tmp-dir\") pod \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") "
Sep 08 20:43:18.943229 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.943225    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b987aeab-4ab2-488d-acfa-2e0704e86a3c-tls-certs\") pod \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") "
Sep 08 20:43:18.943383 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.943312    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzvvj\" (UniqueName: \"kubernetes.io/projected/b987aeab-4ab2-488d-acfa-2e0704e86a3c-kube-api-access-kzvvj\") pod \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") "
Sep 08 20:43:18.943383 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.943337    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-model-cache\") pod \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\" (UID: \"b987aeab-4ab2-488d-acfa-2e0704e86a3c\") "
Sep 08 20:43:18.943383 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.943342    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-home" (OuterVolumeSpecName: "home") pod "b987aeab-4ab2-488d-acfa-2e0704e86a3c" (UID: "b987aeab-4ab2-488d-acfa-2e0704e86a3c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:18.943539 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.943494    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b987aeab-4ab2-488d-acfa-2e0704e86a3c" (UID: "b987aeab-4ab2-488d-acfa-2e0704e86a3c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:18.943661 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.943638    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-model-cache" (OuterVolumeSpecName: "model-cache") pod "b987aeab-4ab2-488d-acfa-2e0704e86a3c" (UID: "b987aeab-4ab2-488d-acfa-2e0704e86a3c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:18.943758 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.943743    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-model-cache\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:18.943811 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.943759    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-home\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:18.943811 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.943768    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-tmp-dir\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:18.945439 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.945414    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-dshm" (OuterVolumeSpecName: "dshm") pod "b987aeab-4ab2-488d-acfa-2e0704e86a3c" (UID: "b987aeab-4ab2-488d-acfa-2e0704e86a3c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:18.945540 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.945463    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b987aeab-4ab2-488d-acfa-2e0704e86a3c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b987aeab-4ab2-488d-acfa-2e0704e86a3c" (UID: "b987aeab-4ab2-488d-acfa-2e0704e86a3c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:43:18.945540 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.945505    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b987aeab-4ab2-488d-acfa-2e0704e86a3c-kube-api-access-kzvvj" (OuterVolumeSpecName: "kube-api-access-kzvvj") pod "b987aeab-4ab2-488d-acfa-2e0704e86a3c" (UID: "b987aeab-4ab2-488d-acfa-2e0704e86a3c"). InnerVolumeSpecName "kube-api-access-kzvvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:43:18.995887 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.995809    2580 generic.go:356] "Generic (PLEG): container finished" podID="b987aeab-4ab2-488d-acfa-2e0704e86a3c" containerID="cc2e7ac5db84690d7fffc744143dda3aa166f53d7970cfa32838e579f5b55b80" exitCode=0
Sep 08 20:43:18.995887 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.995853    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"
Sep 08 20:43:18.996130 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.995896    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl" event={"ID":"b987aeab-4ab2-488d-acfa-2e0704e86a3c","Type":"ContainerDied","Data":"cc2e7ac5db84690d7fffc744143dda3aa166f53d7970cfa32838e579f5b55b80"}
Sep 08 20:43:18.996130 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.995948    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl" event={"ID":"b987aeab-4ab2-488d-acfa-2e0704e86a3c","Type":"ContainerDied","Data":"cdfe2d6ee77a91a9bc11e942349f0ba5ad435cf3b5c6bd35570bc270d6c20c8b"}
Sep 08 20:43:18.996130 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:18.995998    2580 scope.go:117] "RemoveContainer" containerID="cc2e7ac5db84690d7fffc744143dda3aa166f53d7970cfa32838e579f5b55b80"
Sep 08 20:43:19.005820 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:19.005802    2580 scope.go:117] "RemoveContainer" containerID="cc2e7ac5db84690d7fffc744143dda3aa166f53d7970cfa32838e579f5b55b80"
Sep 08 20:43:19.006118 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:43:19.006096    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc2e7ac5db84690d7fffc744143dda3aa166f53d7970cfa32838e579f5b55b80\": container with ID starting with cc2e7ac5db84690d7fffc744143dda3aa166f53d7970cfa32838e579f5b55b80 not found: ID does not exist" containerID="cc2e7ac5db84690d7fffc744143dda3aa166f53d7970cfa32838e579f5b55b80"
Sep 08 20:43:19.006157 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:19.006125    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc2e7ac5db84690d7fffc744143dda3aa166f53d7970cfa32838e579f5b55b80"} err="failed to get container status \"cc2e7ac5db84690d7fffc744143dda3aa166f53d7970cfa32838e579f5b55b80\": rpc error: code = NotFound desc = could not find container \"cc2e7ac5db84690d7fffc744143dda3aa166f53d7970cfa32838e579f5b55b80\": container with ID starting with cc2e7ac5db84690d7fffc744143dda3aa166f53d7970cfa32838e579f5b55b80 not found: ID does not exist"
Sep 08 20:43:19.018775 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:19.018742    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"]
Sep 08 20:43:19.021459 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:19.021427    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:19.021716 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:19.021696    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-6f8575/test-model-6f8575-kserve-74765d97bc-kt4pl"]
Sep 08 20:43:19.044627 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:19.044592    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b987aeab-4ab2-488d-acfa-2e0704e86a3c-dshm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:19.044627 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:19.044623    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b987aeab-4ab2-488d-acfa-2e0704e86a3c-tls-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:19.044798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:19.044708    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzvvj\" (UniqueName: \"kubernetes.io/projected/b987aeab-4ab2-488d-acfa-2e0704e86a3c-kube-api-access-kzvvj\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:19.593912 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:19.593879    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:19.699862 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:19.694484    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:19.780711 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:19.780675    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:19.834895 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:19.834864    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:20.123169 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.123140    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:20.199723 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.199680    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n" podUID="8c40aad2-190c-4d0b-8e20-df0ef325fcdd" containerName="main" containerID="cri-o://382803a0d4daafa981d1fd66f9c957dfcf2b0c0a82e8c7dd52ee99f4f7c467b6" gracePeriod=45
Sep 08 20:43:20.445215 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.445185    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:43:20.560456 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.560417    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-home\") pod \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") "
Sep 08 20:43:20.560456 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.560460    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-dshm\") pod \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") "
Sep 08 20:43:20.560708 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.560513    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-tls-certs\") pod \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") "
Sep 08 20:43:20.560708 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.560574    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-model-cache\") pod \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") "
Sep 08 20:43:20.560708 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.560630    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2p89\" (UniqueName: \"kubernetes.io/projected/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-kube-api-access-c2p89\") pod \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") "
Sep 08 20:43:20.560708 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.560687    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-tmp-dir\") pod \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\" (UID: \"8c40aad2-190c-4d0b-8e20-df0ef325fcdd\") "
Sep 08 20:43:20.560923 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.560753    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-home" (OuterVolumeSpecName: "home") pod "8c40aad2-190c-4d0b-8e20-df0ef325fcdd" (UID: "8c40aad2-190c-4d0b-8e20-df0ef325fcdd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:20.560923 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.560904    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-model-cache" (OuterVolumeSpecName: "model-cache") pod "8c40aad2-190c-4d0b-8e20-df0ef325fcdd" (UID: "8c40aad2-190c-4d0b-8e20-df0ef325fcdd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:20.561137 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.560944    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8c40aad2-190c-4d0b-8e20-df0ef325fcdd" (UID: "8c40aad2-190c-4d0b-8e20-df0ef325fcdd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:20.561137 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.561130    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-home\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:20.561251 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.561142    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-model-cache\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:20.561251 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.561151    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-tmp-dir\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:20.562797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.562769    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-dshm" (OuterVolumeSpecName: "dshm") pod "8c40aad2-190c-4d0b-8e20-df0ef325fcdd" (UID: "8c40aad2-190c-4d0b-8e20-df0ef325fcdd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:20.562928 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.562820    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8c40aad2-190c-4d0b-8e20-df0ef325fcdd" (UID: "8c40aad2-190c-4d0b-8e20-df0ef325fcdd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:43:20.562928 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.562841    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-kube-api-access-c2p89" (OuterVolumeSpecName: "kube-api-access-c2p89") pod "8c40aad2-190c-4d0b-8e20-df0ef325fcdd" (UID: "8c40aad2-190c-4d0b-8e20-df0ef325fcdd"). InnerVolumeSpecName "kube-api-access-c2p89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:43:20.661732 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.661695    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2p89\" (UniqueName: \"kubernetes.io/projected/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-kube-api-access-c2p89\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:20.661732 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.661729    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-dshm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:20.661732 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.661740    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c40aad2-190c-4d0b-8e20-df0ef325fcdd-tls-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:20.924664 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:20.924620    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b987aeab-4ab2-488d-acfa-2e0704e86a3c" path="/var/lib/kubelet/pods/b987aeab-4ab2-488d-acfa-2e0704e86a3c/volumes"
Sep 08 20:43:21.007581 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.007542    2580 generic.go:356] "Generic (PLEG): container finished" podID="8c40aad2-190c-4d0b-8e20-df0ef325fcdd" containerID="382803a0d4daafa981d1fd66f9c957dfcf2b0c0a82e8c7dd52ee99f4f7c467b6" exitCode=0
Sep 08 20:43:21.007744 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.007592    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"
Sep 08 20:43:21.007744 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.007627    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n" event={"ID":"8c40aad2-190c-4d0b-8e20-df0ef325fcdd","Type":"ContainerDied","Data":"382803a0d4daafa981d1fd66f9c957dfcf2b0c0a82e8c7dd52ee99f4f7c467b6"}
Sep 08 20:43:21.007744 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.007672    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n" event={"ID":"8c40aad2-190c-4d0b-8e20-df0ef325fcdd","Type":"ContainerDied","Data":"2ebb09ca79785c9de810f0df73516cf50982cbb1440b193fa2e92842fe2d0a7d"}
Sep 08 20:43:21.007744 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.007695    2580 scope.go:117] "RemoveContainer" containerID="382803a0d4daafa981d1fd66f9c957dfcf2b0c0a82e8c7dd52ee99f4f7c467b6"
Sep 08 20:43:21.016339 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.016319    2580 scope.go:117] "RemoveContainer" containerID="382803a0d4daafa981d1fd66f9c957dfcf2b0c0a82e8c7dd52ee99f4f7c467b6"
Sep 08 20:43:21.016656 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:43:21.016635    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"382803a0d4daafa981d1fd66f9c957dfcf2b0c0a82e8c7dd52ee99f4f7c467b6\": container with ID starting with 382803a0d4daafa981d1fd66f9c957dfcf2b0c0a82e8c7dd52ee99f4f7c467b6 not found: ID does not exist" containerID="382803a0d4daafa981d1fd66f9c957dfcf2b0c0a82e8c7dd52ee99f4f7c467b6"
Sep 08 20:43:21.016709 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.016664    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"382803a0d4daafa981d1fd66f9c957dfcf2b0c0a82e8c7dd52ee99f4f7c467b6"} err="failed to get container status \"382803a0d4daafa981d1fd66f9c957dfcf2b0c0a82e8c7dd52ee99f4f7c467b6\": rpc error: code = NotFound desc = could not find container \"382803a0d4daafa981d1fd66f9c957dfcf2b0c0a82e8c7dd52ee99f4f7c467b6\": container with ID starting with 382803a0d4daafa981d1fd66f9c957dfcf2b0c0a82e8c7dd52ee99f4f7c467b6 not found: ID does not exist"
Sep 08 20:43:21.023797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.023766    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"]
Sep 08 20:43:21.027627 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.027604    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-458e34/noten-model-458e34-kserve-7dc58859ff-tsh4n"]
Sep 08 20:43:21.198356 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.198273    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w" podUID="5ff2c5bd-5eb0-4312-93a2-06ad2231d55b" containerName="main" containerID="cri-o://cde95f77ba593213aedcb50672cc88a53cdd8e6969e690724aa8a2f1f81dc031" gracePeriod=45
Sep 08 20:43:21.454095 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.453926    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:43:21.570266 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.570228    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-tmp-dir\") pod \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") "
Sep 08 20:43:21.570480 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.570316    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-tls-certs\") pod \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") "
Sep 08 20:43:21.570480 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.570375    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-dshm\") pod \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") "
Sep 08 20:43:21.570480 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.570425    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-model-cache\") pod \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") "
Sep 08 20:43:21.570480 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.570450    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-home\") pod \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") "
Sep 08 20:43:21.570704 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.570488    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8hfz\" (UniqueName: \"kubernetes.io/projected/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-kube-api-access-v8hfz\") pod \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\" (UID: \"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b\") "
Sep 08 20:43:21.570704 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.570563    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5ff2c5bd-5eb0-4312-93a2-06ad2231d55b" (UID: "5ff2c5bd-5eb0-4312-93a2-06ad2231d55b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:21.570798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.570722    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-home" (OuterVolumeSpecName: "home") pod "5ff2c5bd-5eb0-4312-93a2-06ad2231d55b" (UID: "5ff2c5bd-5eb0-4312-93a2-06ad2231d55b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:21.570798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.570718    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-model-cache" (OuterVolumeSpecName: "model-cache") pod "5ff2c5bd-5eb0-4312-93a2-06ad2231d55b" (UID: "5ff2c5bd-5eb0-4312-93a2-06ad2231d55b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:21.570798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.570778    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-tmp-dir\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:21.572761 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.572731    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-dshm" (OuterVolumeSpecName: "dshm") pod "5ff2c5bd-5eb0-4312-93a2-06ad2231d55b" (UID: "5ff2c5bd-5eb0-4312-93a2-06ad2231d55b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:21.572862 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.572772    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5ff2c5bd-5eb0-4312-93a2-06ad2231d55b" (UID: "5ff2c5bd-5eb0-4312-93a2-06ad2231d55b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:43:21.572862 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.572808    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-kube-api-access-v8hfz" (OuterVolumeSpecName: "kube-api-access-v8hfz") pod "5ff2c5bd-5eb0-4312-93a2-06ad2231d55b" (UID: "5ff2c5bd-5eb0-4312-93a2-06ad2231d55b"). InnerVolumeSpecName "kube-api-access-v8hfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:43:21.672184 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.672147    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-tls-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:21.672184 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.672177    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-dshm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:21.672184 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.672186    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-model-cache\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:21.672184 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.672193    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-home\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:21.672452 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:21.672201    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8hfz\" (UniqueName: \"kubernetes.io/projected/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b-kube-api-access-v8hfz\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:22.013897 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:22.013860    2580 generic.go:356] "Generic (PLEG): container finished" podID="5ff2c5bd-5eb0-4312-93a2-06ad2231d55b" containerID="cde95f77ba593213aedcb50672cc88a53cdd8e6969e690724aa8a2f1f81dc031" exitCode=0
Sep 08 20:43:22.014397 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:22.013908    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"
Sep 08 20:43:22.014397 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:22.013939    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w" event={"ID":"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b","Type":"ContainerDied","Data":"cde95f77ba593213aedcb50672cc88a53cdd8e6969e690724aa8a2f1f81dc031"}
Sep 08 20:43:22.014397 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:22.014006    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w" event={"ID":"5ff2c5bd-5eb0-4312-93a2-06ad2231d55b","Type":"ContainerDied","Data":"6b56108aeb06a4b1578b59de247a4f901ce6f5e5b0ea8a897a26e0aeab26473d"}
Sep 08 20:43:22.014397 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:22.014029    2580 scope.go:117] "RemoveContainer" containerID="cde95f77ba593213aedcb50672cc88a53cdd8e6969e690724aa8a2f1f81dc031"
Sep 08 20:43:22.023245 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:22.023229    2580 scope.go:117] "RemoveContainer" containerID="cde95f77ba593213aedcb50672cc88a53cdd8e6969e690724aa8a2f1f81dc031"
Sep 08 20:43:22.023515 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:43:22.023493    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cde95f77ba593213aedcb50672cc88a53cdd8e6969e690724aa8a2f1f81dc031\": container with ID starting with cde95f77ba593213aedcb50672cc88a53cdd8e6969e690724aa8a2f1f81dc031 not found: ID does not exist" containerID="cde95f77ba593213aedcb50672cc88a53cdd8e6969e690724aa8a2f1f81dc031"
Sep 08 20:43:22.023565 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:22.023526    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cde95f77ba593213aedcb50672cc88a53cdd8e6969e690724aa8a2f1f81dc031"} err="failed to get container status \"cde95f77ba593213aedcb50672cc88a53cdd8e6969e690724aa8a2f1f81dc031\": rpc error: code = NotFound desc = could not find container \"cde95f77ba593213aedcb50672cc88a53cdd8e6969e690724aa8a2f1f81dc031\": container with ID starting with cde95f77ba593213aedcb50672cc88a53cdd8e6969e690724aa8a2f1f81dc031 not found: ID does not exist"
Sep 08 20:43:22.034673 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:22.034651    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"]
Sep 08 20:43:22.038238 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:22.038207    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-a6137b/test-model-a6137b-kserve-6fb68876cc-bg67w"]
Sep 08 20:43:22.923928 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:22.923893    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ff2c5bd-5eb0-4312-93a2-06ad2231d55b" path="/var/lib/kubelet/pods/5ff2c5bd-5eb0-4312-93a2-06ad2231d55b/volumes"
Sep 08 20:43:22.924346 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:22.924329    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c40aad2-190c-4d0b-8e20-df0ef325fcdd" path="/var/lib/kubelet/pods/8c40aad2-190c-4d0b-8e20-df0ef325fcdd/volumes"
Sep 08 20:43:23.580841 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:23.580803    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:23.643782 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:23.643746    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:23.847866 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:23.847778    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:24.324532 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:24.324495    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:26.386614 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:26.386564    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:26.452902 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:26.452865    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:26.625285 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:26.625248    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:26.947701 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:26.947666    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:27.046518 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:27.046480    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:27.662506 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:27.662474    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:32.863676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:32.863639    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:32.934407 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:32.934375    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:33.019125 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:33.019091    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:33.210097 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:33.210058    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:34.644300 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:34.644270    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"]
Sep 08 20:43:34.644719 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:34.644485    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6" podUID="1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4" containerName="maas-api" containerID="cri-o://301be47488830a53635d9a5ea474b07f9231316896c561d45b756c42542317ce" gracePeriod=30
Sep 08 20:43:34.673767 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:34.673707    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:34.788348 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:34.788305    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:34.884978 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:34.884936    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:34.901784 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:34.901725    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:43:34.985973 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:34.985934    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:34.996167 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:34.996144    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-maas-api-tls\") pod \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") "
Sep 08 20:43:34.996252 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:34.996240    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rg6l\" (UniqueName: \"kubernetes.io/projected/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-kube-api-access-7rg6l\") pod \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") "
Sep 08 20:43:34.996292 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:34.996266    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-metrics-certs\") pod \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") "
Sep 08 20:43:34.996347 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:34.996291    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-openshift-service-ca\") pod \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\" (UID: \"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4\") "
Sep 08 20:43:34.996705 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:34.996675    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4" (UID: "1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:43:34.998513 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:34.998485    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-kube-api-access-7rg6l" (OuterVolumeSpecName: "kube-api-access-7rg6l") pod "1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4" (UID: "1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4"). InnerVolumeSpecName "kube-api-access-7rg6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:43:34.998624 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:34.998517    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4" (UID: "1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:43:34.998624 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:34.998531    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4" (UID: "1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:43:35.077303 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:35.077271    2580 generic.go:356] "Generic (PLEG): container finished" podID="1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4" containerID="301be47488830a53635d9a5ea474b07f9231316896c561d45b756c42542317ce" exitCode=0
Sep 08 20:43:35.077465 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:35.077335    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"
Sep 08 20:43:35.077465 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:35.077351    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6" event={"ID":"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4","Type":"ContainerDied","Data":"301be47488830a53635d9a5ea474b07f9231316896c561d45b756c42542317ce"}
Sep 08 20:43:35.077465 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:35.077408    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6" event={"ID":"1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4","Type":"ContainerDied","Data":"64136aac637e9ede2274dad97b0615cd68ae785adadcc3e43a029f0473662a14"}
Sep 08 20:43:35.077465 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:35.077436    2580 scope.go:117] "RemoveContainer" containerID="301be47488830a53635d9a5ea474b07f9231316896c561d45b756c42542317ce"
Sep 08 20:43:35.089239 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:35.089212    2580 scope.go:117] "RemoveContainer" containerID="301be47488830a53635d9a5ea474b07f9231316896c561d45b756c42542317ce"
Sep 08 20:43:35.089548 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:43:35.089527    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"301be47488830a53635d9a5ea474b07f9231316896c561d45b756c42542317ce\": container with ID starting with 301be47488830a53635d9a5ea474b07f9231316896c561d45b756c42542317ce not found: ID does not exist" containerID="301be47488830a53635d9a5ea474b07f9231316896c561d45b756c42542317ce"
Sep 08 20:43:35.089611 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:35.089557    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"301be47488830a53635d9a5ea474b07f9231316896c561d45b756c42542317ce"} err="failed to get container status \"301be47488830a53635d9a5ea474b07f9231316896c561d45b756c42542317ce\": rpc error: code = NotFound desc = could not find container \"301be47488830a53635d9a5ea474b07f9231316896c561d45b756c42542317ce\": container with ID starting with 301be47488830a53635d9a5ea474b07f9231316896c561d45b756c42542317ce not found: ID does not exist"
Sep 08 20:43:35.097893 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:35.097867    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rg6l\" (UniqueName: \"kubernetes.io/projected/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-kube-api-access-7rg6l\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:35.097893 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:35.097892    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-metrics-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:35.098071 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:35.097904    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-openshift-service-ca\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:35.098071 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:35.097912    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4-maas-api-tls\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:35.105568 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:35.105545    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"]
Sep 08 20:43:35.110827 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:35.110806    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a6137b-6cf549b7cc-6tsj6"]
Sep 08 20:43:35.271818 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:35.271737    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:36.924012 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:36.923969    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4" path="/var/lib/kubelet/pods/1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4/volumes"
Sep 08 20:43:38.943949 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:38.943917    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:39.008179 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:39.008145    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:39.837380 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:39.837325    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:40.149746 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:40.149710    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:40.348719 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:40.348683    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:41.393940 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:41.393909    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:41.463999 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:41.463949    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:41.617728 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:41.617686    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"]
Sep 08 20:43:41.617986 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:41.617944    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm" podUID="1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" containerName="istio-proxy" containerID="cri-o://2e48c596c1deaa979a6ee4d14dc407402c5f51d2c34350fe2f40a45f45320928" gracePeriod=30
Sep 08 20:43:41.646442 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:41.646356    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:42.196952 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.196915    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:42.468079 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.467988    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"]
Sep 08 20:43:42.468528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.468340    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c40aad2-190c-4d0b-8e20-df0ef325fcdd" containerName="main"
Sep 08 20:43:42.468528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.468351    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c40aad2-190c-4d0b-8e20-df0ef325fcdd" containerName="main"
Sep 08 20:43:42.468528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.468370    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b705417-23b4-4c43-a46d-cf5c59b3fc52" containerName="maas-api"
Sep 08 20:43:42.468528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.468376    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b705417-23b4-4c43-a46d-cf5c59b3fc52" containerName="maas-api"
Sep 08 20:43:42.468528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.468383    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4" containerName="maas-api"
Sep 08 20:43:42.468528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.468389    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4" containerName="maas-api"
Sep 08 20:43:42.468528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.468399    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b987aeab-4ab2-488d-acfa-2e0704e86a3c" containerName="main"
Sep 08 20:43:42.468528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.468404    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b987aeab-4ab2-488d-acfa-2e0704e86a3c" containerName="main"
Sep 08 20:43:42.468528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.468411    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ff2c5bd-5eb0-4312-93a2-06ad2231d55b" containerName="main"
Sep 08 20:43:42.468528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.468416    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ff2c5bd-5eb0-4312-93a2-06ad2231d55b" containerName="main"
Sep 08 20:43:42.468528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.468468    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b705417-23b4-4c43-a46d-cf5c59b3fc52" containerName="maas-api"
Sep 08 20:43:42.468528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.468474    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ff2c5bd-5eb0-4312-93a2-06ad2231d55b" containerName="main"
Sep 08 20:43:42.468528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.468484    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b987aeab-4ab2-488d-acfa-2e0704e86a3c" containerName="main"
Sep 08 20:43:42.468528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.468490    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e82ddb9-ff2b-41c7-bf4d-1bb1e0e854c4" containerName="maas-api"
Sep 08 20:43:42.468528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.468498    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c40aad2-190c-4d0b-8e20-df0ef325fcdd" containerName="main"
Sep 08 20:43:42.472775 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.472753    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.474851 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.474833    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-78dd9f-openshift-default-dockercfg-s6nj2\""
Sep 08 20:43:42.485189 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.485163    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"]
Sep 08 20:43:42.509864 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.509831    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:42.570025 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.569990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-podinfo\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.570208 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.570040    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-workload-socket\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.570208 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.570118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istiod-ca-cert\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.570208 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.570164    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-credential-socket\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.570208 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.570189    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-workload-certs\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.570208 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.570206    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-envoy\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.570383 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.570255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-token\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.570383 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.570284    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hzdw\" (UniqueName: \"kubernetes.io/projected/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-kube-api-access-7hzdw\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.570383 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.570314    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-data\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.671700 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.671655    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-data\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.671899 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.671718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-podinfo\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.671899 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.671764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-workload-socket\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.671899 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.671815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istiod-ca-cert\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.671899 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.671847    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-credential-socket\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.672155 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.672000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-workload-certs\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.672155 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.672064    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-envoy\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.672155 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.672112    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-token\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.672325 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.672153    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hzdw\" (UniqueName: \"kubernetes.io/projected/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-kube-api-access-7hzdw\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.672325 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.672165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-data\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.672325 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.672193    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-workload-socket\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.672510 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.672356    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-workload-certs\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.672510 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.672395    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-credential-socket\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.672624 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.672569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istiod-ca-cert\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.674301 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.674279    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-envoy\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.674535 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.674516    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-podinfo\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.679755 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.679730    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hzdw\" (UniqueName: \"kubernetes.io/projected/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-kube-api-access-7hzdw\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.679896 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.679875    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-token\") pod \"e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") " pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:42.784179 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:42.784086    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:43.061874 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:43.061845    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"]
Sep 08 20:43:43.064535 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:43:43.064503    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe4d6ee4_d36e_4929_b356_15fde3dde9aa.slice/crio-6dbee8e9e73cb536d3902774a42f8f0b950cb1fd6266f7eac98ebd0f8f4a5e53 WatchSource:0}: Error finding container 6dbee8e9e73cb536d3902774a42f8f0b950cb1fd6266f7eac98ebd0f8f4a5e53: Status 404 returned error can't find the container with id 6dbee8e9e73cb536d3902774a42f8f0b950cb1fd6266f7eac98ebd0f8f4a5e53
Sep 08 20:43:43.066829 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:43.066795    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:43:43.066926 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:43.066859    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:43:43.066926 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:43.066887    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:43:43.119367 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:43.119339    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h" event={"ID":"fe4d6ee4-d36e-4929-b356-15fde3dde9aa","Type":"ContainerStarted","Data":"6dbee8e9e73cb536d3902774a42f8f0b950cb1fd6266f7eac98ebd0f8f4a5e53"}
Sep 08 20:43:43.144466 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:43.144438    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:43.251422 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:43.251377    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:43.344937 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:43.344844    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:44.126383 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:44.126353    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h" event={"ID":"fe4d6ee4-d36e-4929-b356-15fde3dde9aa","Type":"ContainerStarted","Data":"c45f09717675550df07a3d81dbe1265ef005d0955b8c2604ada5d2e78911ee84"}
Sep 08 20:43:44.150221 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:44.150169    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h" podStartSLOduration=2.15014598 podStartE2EDuration="2.15014598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:43:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:43:44.148612915 +0000 UTC m=+1097.695862410" watchObservedRunningTime="2026-09-08 20:43:44.15014598 +0000 UTC m=+1097.697395460"
Sep 08 20:43:44.542028 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:44.541996    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:44.606376 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:44.606342    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:44.706727 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:44.706693    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:44.785224 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:44.785186    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:44.790679 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:44.790645    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:45.103220 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:45.103189    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:45.131431 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:45.131402    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:45.132457 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:45.132436    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:43:45.187994 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:45.187950    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:45.474751 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:45.474711    2580 patch_prober.go:28] interesting pod/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 20:43:45.474924 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:45.474774    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm" podUID="1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 20:43:46.099883 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.099838    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:46.550578 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.550543    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:46.869031 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.869006    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:43:46.922215 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.922186    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-workload-socket\") pod \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") "
Sep 08 20:43:46.922432 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.922237    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-workload-certs\") pod \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") "
Sep 08 20:43:46.922432 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.922255    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istiod-ca-cert\") pod \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") "
Sep 08 20:43:46.922432 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.922271    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-credential-socket\") pod \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") "
Sep 08 20:43:46.922432 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.922316    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-envoy\") pod \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") "
Sep 08 20:43:46.922432 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.922389    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-podinfo\") pod \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") "
Sep 08 20:43:46.922432 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.922428    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qn64v\" (UniqueName: \"kubernetes.io/projected/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-kube-api-access-qn64v\") pod \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") "
Sep 08 20:43:46.922749 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.922434    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" (UID: "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:46.922749 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.922477    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-data\") pod \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") "
Sep 08 20:43:46.922749 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.922521    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-token\") pod \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\" (UID: \"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211\") "
Sep 08 20:43:46.922749 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.922562    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" (UID: "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:46.922749 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.922638    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" (UID: "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:43:46.923033 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.922814    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-data" (OuterVolumeSpecName: "istio-data") pod "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" (UID: "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:46.923033 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.922887    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" (UID: "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:46.923033 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.922994    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-workload-socket\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:46.923033 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.923014    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-workload-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:46.923033 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.923028    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istiod-ca-cert\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:46.923282 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.923042    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-credential-socket\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:46.923282 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.923055    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-data\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:46.924623 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.924591    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" (UID: "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:43:46.925041 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.925017    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" (UID: "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:43:46.925119 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.925035    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-token" (OuterVolumeSpecName: "istio-token") pod "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" (UID: "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:43:46.925119 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:46.925080    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-kube-api-access-qn64v" (OuterVolumeSpecName: "kube-api-access-qn64v") pod "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" (UID: "1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211"). InnerVolumeSpecName "kube-api-access-qn64v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:43:47.023615 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.023571    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-token\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:47.023615 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.023614    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-envoy\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:47.023824 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.023627    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-istio-podinfo\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:47.023824 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.023641    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qn64v\" (UniqueName: \"kubernetes.io/projected/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211-kube-api-access-qn64v\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:43:47.142981 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.142921    2580 generic.go:356] "Generic (PLEG): container finished" podID="1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" containerID="2e48c596c1deaa979a6ee4d14dc407402c5f51d2c34350fe2f40a45f45320928" exitCode=0
Sep 08 20:43:47.142981 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.142978    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"
Sep 08 20:43:47.143210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.143002    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm" event={"ID":"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211","Type":"ContainerDied","Data":"2e48c596c1deaa979a6ee4d14dc407402c5f51d2c34350fe2f40a45f45320928"}
Sep 08 20:43:47.143210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.143055    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm" event={"ID":"1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211","Type":"ContainerDied","Data":"acba6fef45613c2deb6cd9514708ac26570ae8ad2871279f7f85aef884aef236"}
Sep 08 20:43:47.143210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.143081    2580 scope.go:117] "RemoveContainer" containerID="2e48c596c1deaa979a6ee4d14dc407402c5f51d2c34350fe2f40a45f45320928"
Sep 08 20:43:47.152040 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.152021    2580 scope.go:117] "RemoveContainer" containerID="7d0dfed4c219fa7c4d092a6a2ba933f236843126e736b1b5d4e5fd975d3a0b16"
Sep 08 20:43:47.159947 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.159930    2580 scope.go:117] "RemoveContainer" containerID="2e48c596c1deaa979a6ee4d14dc407402c5f51d2c34350fe2f40a45f45320928"
Sep 08 20:43:47.160221 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:43:47.160198    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e48c596c1deaa979a6ee4d14dc407402c5f51d2c34350fe2f40a45f45320928\": container with ID starting with 2e48c596c1deaa979a6ee4d14dc407402c5f51d2c34350fe2f40a45f45320928 not found: ID does not exist" containerID="2e48c596c1deaa979a6ee4d14dc407402c5f51d2c34350fe2f40a45f45320928"
Sep 08 20:43:47.160269 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.160228    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e48c596c1deaa979a6ee4d14dc407402c5f51d2c34350fe2f40a45f45320928"} err="failed to get container status \"2e48c596c1deaa979a6ee4d14dc407402c5f51d2c34350fe2f40a45f45320928\": rpc error: code = NotFound desc = could not find container \"2e48c596c1deaa979a6ee4d14dc407402c5f51d2c34350fe2f40a45f45320928\": container with ID starting with 2e48c596c1deaa979a6ee4d14dc407402c5f51d2c34350fe2f40a45f45320928 not found: ID does not exist"
Sep 08 20:43:47.160269 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.160252    2580 scope.go:117] "RemoveContainer" containerID="7d0dfed4c219fa7c4d092a6a2ba933f236843126e736b1b5d4e5fd975d3a0b16"
Sep 08 20:43:47.160485 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:43:47.160465    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d0dfed4c219fa7c4d092a6a2ba933f236843126e736b1b5d4e5fd975d3a0b16\": container with ID starting with 7d0dfed4c219fa7c4d092a6a2ba933f236843126e736b1b5d4e5fd975d3a0b16 not found: ID does not exist" containerID="7d0dfed4c219fa7c4d092a6a2ba933f236843126e736b1b5d4e5fd975d3a0b16"
Sep 08 20:43:47.160556 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.160490    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d0dfed4c219fa7c4d092a6a2ba933f236843126e736b1b5d4e5fd975d3a0b16"} err="failed to get container status \"7d0dfed4c219fa7c4d092a6a2ba933f236843126e736b1b5d4e5fd975d3a0b16\": rpc error: code = NotFound desc = could not find container \"7d0dfed4c219fa7c4d092a6a2ba933f236843126e736b1b5d4e5fd975d3a0b16\": container with ID starting with 7d0dfed4c219fa7c4d092a6a2ba933f236843126e736b1b5d4e5fd975d3a0b16 not found: ID does not exist"
Sep 08 20:43:47.170543 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.170515    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"]
Sep 08 20:43:47.176768 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.176743    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-a6137b-openshift-default-7bf5895688-sgxvm"]
Sep 08 20:43:47.230068 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.230027    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"]
Sep 08 20:43:47.230935 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.230909    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" containerName="istio-proxy"
Sep 08 20:43:47.230935 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.230935    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" containerName="istio-proxy"
Sep 08 20:43:47.231174 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.230987    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" containerName="istio-proxy"
Sep 08 20:43:47.231174 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.230996    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" containerName="istio-proxy"
Sep 08 20:43:47.231174 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.231150    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" containerName="istio-proxy"
Sep 08 20:43:47.231174 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.231170    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" containerName="istio-proxy"
Sep 08 20:43:47.239156 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.239124    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:43:47.241484 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.241454    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-78dd9f-serving-cert\""
Sep 08 20:43:47.242685 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.242653    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"]
Sep 08 20:43:47.268614 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.268571    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:47.326656 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.326620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/124d18f2-8058-4281-8d27-748adbd924ab-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:43:47.326830 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.326679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94ng5\" (UniqueName: \"kubernetes.io/projected/124d18f2-8058-4281-8d27-748adbd924ab-kube-api-access-94ng5\") pod \"maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:43:47.326830 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.326706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/124d18f2-8058-4281-8d27-748adbd924ab-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:43:47.326830 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.326749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/124d18f2-8058-4281-8d27-748adbd924ab-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:43:47.369242 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.369209    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:47.427658 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.427560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/124d18f2-8058-4281-8d27-748adbd924ab-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:43:47.427658 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.427611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94ng5\" (UniqueName: \"kubernetes.io/projected/124d18f2-8058-4281-8d27-748adbd924ab-kube-api-access-94ng5\") pod \"maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:43:47.427658 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.427637    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/124d18f2-8058-4281-8d27-748adbd924ab-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:43:47.427944 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.427665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/124d18f2-8058-4281-8d27-748adbd924ab-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:43:47.427944 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:43:47.427788    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-serving-cert: secret "maas-api-e2e-shared-a-w3-78dd9f-serving-cert" not found
Sep 08 20:43:47.427944 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:43:47.427864    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/124d18f2-8058-4281-8d27-748adbd924ab-maas-api-tls podName:124d18f2-8058-4281-8d27-748adbd924ab nodeName:}" failed. No retries permitted until 2026-09-08 20:43:47.927842984 +0000 UTC m=+1101.475092447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/124d18f2-8058-4281-8d27-748adbd924ab-maas-api-tls") pod "maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb" (UID: "124d18f2-8058-4281-8d27-748adbd924ab") : secret "maas-api-e2e-shared-a-w3-78dd9f-serving-cert" not found
Sep 08 20:43:47.428378 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.428357    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/124d18f2-8058-4281-8d27-748adbd924ab-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:43:47.430195 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.430174    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/124d18f2-8058-4281-8d27-748adbd924ab-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:43:47.436569 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.436547    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94ng5\" (UniqueName: \"kubernetes.io/projected/124d18f2-8058-4281-8d27-748adbd924ab-kube-api-access-94ng5\") pod \"maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:43:47.741765 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.741670    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:47.839905 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.839875    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:47.932869 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.932826    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/124d18f2-8058-4281-8d27-748adbd924ab-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:43:47.935632 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.935607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/124d18f2-8058-4281-8d27-748adbd924ab-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:43:47.944597 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:47.944568    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:48.151363 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:48.151330    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:43:48.321253 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:48.321220    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:43:48.424454 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:48.424428    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"]
Sep 08 20:43:48.426659 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:43:48.426633    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod124d18f2_8058_4281_8d27_748adbd924ab.slice/crio-1ffe37c9186b93aa09478340cbefc96db4d5719a66394d07db060965b7c30432 WatchSource:0}: Error finding container 1ffe37c9186b93aa09478340cbefc96db4d5719a66394d07db060965b7c30432: Status 404 returned error can't find the container with id 1ffe37c9186b93aa09478340cbefc96db4d5719a66394d07db060965b7c30432
Sep 08 20:43:48.427986 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:48.427948    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 20:43:48.924784 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:48.924743    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211" path="/var/lib/kubelet/pods/1f9a1ec4-0aa0-4f5b-95a0-0832ff3b5211/volumes"
Sep 08 20:43:49.158946 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:49.158912    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb" event={"ID":"124d18f2-8058-4281-8d27-748adbd924ab","Type":"ContainerStarted","Data":"022ca8cd7a83ea775b255267b92d79ef21d7a64db9ec9c0a23790e6971530f3e"}
Sep 08 20:43:49.159138 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:49.158974    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb" event={"ID":"124d18f2-8058-4281-8d27-748adbd924ab","Type":"ContainerStarted","Data":"1ffe37c9186b93aa09478340cbefc96db4d5719a66394d07db060965b7c30432"}
Sep 08 20:43:49.159138 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:49.159075    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:43:49.180496 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:49.180449    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb" podStartSLOduration=1.8189027150000001 podStartE2EDuration="2.180431875s" totalImagesPullingTime="361.52916ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:43:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:43:49.178730418 +0000 UTC m=+1102.725979903" watchObservedRunningTime="2026-09-08 20:43:49.180431875 +0000 UTC m=+1102.727681356"
Sep 08 20:43:55.169977 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:43:55.169932    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:44:01.173158 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:01.173113    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 08 20:44:01.177226 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:01.173197    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h" podUID="fe4d6ee4-d36e-4929-b356-15fde3dde9aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 08 20:44:02.222931 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:02.222898    2580 generic.go:356] "Generic (PLEG): container finished" podID="fe4d6ee4-d36e-4929-b356-15fde3dde9aa" containerID="c45f09717675550df07a3d81dbe1265ef005d0955b8c2604ada5d2e78911ee84" exitCode=0
Sep 08 20:44:02.223317 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:02.222985    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h" event={"ID":"fe4d6ee4-d36e-4929-b356-15fde3dde9aa","Type":"ContainerDied","Data":"c45f09717675550df07a3d81dbe1265ef005d0955b8c2604ada5d2e78911ee84"}
Sep 08 20:44:02.223457 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:02.223439    2580 scope.go:117] "RemoveContainer" containerID="c45f09717675550df07a3d81dbe1265ef005d0955b8c2604ada5d2e78911ee84"
Sep 08 20:44:02.224589 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:02.224564    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:44:02.224650 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:02.224627    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:44:02.224686 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:02.224656    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:44:03.235830 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:03.235791    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h" event={"ID":"fe4d6ee4-d36e-4929-b356-15fde3dde9aa","Type":"ContainerStarted","Data":"4158ddb88378c0c6a0a213c7a6a66bfacbc34560966f12e812d14ef4d83e927f"}
Sep 08 20:44:03.785229 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:03.785187    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:44:03.785423 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:03.785396    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 08 20:44:03.785473 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:03.785447    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h" podUID="fe4d6ee4-d36e-4929-b356-15fde3dde9aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 08 20:44:04.784826 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:04.784785    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 08 20:44:04.785357 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:04.784854    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h" podUID="fe4d6ee4-d36e-4929-b356-15fde3dde9aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 08 20:44:05.632101 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:05.632057    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:05.784577 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:05.784536    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 08 20:44:05.784759 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:05.784610    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h" podUID="fe4d6ee4-d36e-4929-b356-15fde3dde9aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 08 20:44:06.784827 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:06.784781    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 08 20:44:06.785234 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:06.784854    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h" podUID="fe4d6ee4-d36e-4929-b356-15fde3dde9aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 08 20:44:07.785023 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:07.784984    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Sep 08 20:44:07.785493 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:07.785064    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h" podUID="fe4d6ee4-d36e-4929-b356-15fde3dde9aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Sep 08 20:44:08.182362 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.182321    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:08.284937 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.284903    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:08.499381 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.499294    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:08.773263 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.773168    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"]
Sep 08 20:44:08.775840 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.775817    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.778528 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.778509    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-74f9a1-openshift-default-dockercfg-4mpzs\""
Sep 08 20:44:08.788538 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.788514    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"]
Sep 08 20:44:08.788864 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.788765    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:44:08.789036 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.789019    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:44:08.789703 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.789686    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:44:08.809705 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.809676    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:08.841759 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.841721    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e28c4038-41cf-46fd-91e3-410d097ca036-istiod-ca-cert\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.841953 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.841763    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-istio-data\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.841953 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.841917    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e28c4038-41cf-46fd-91e3-410d097ca036-istio-podinfo\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.842110 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.841949    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e28c4038-41cf-46fd-91e3-410d097ca036-istio-token\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.842110 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.842040    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-workload-certs\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.842110 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.842085    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zzc5\" (UniqueName: \"kubernetes.io/projected/e28c4038-41cf-46fd-91e3-410d097ca036-kube-api-access-8zzc5\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.842275 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.842127    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-credential-socket\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.842275 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.842167    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-istio-envoy\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.842275 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.842200    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-workload-socket\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.942971 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.942916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e28c4038-41cf-46fd-91e3-410d097ca036-istio-podinfo\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.943152 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.942992    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e28c4038-41cf-46fd-91e3-410d097ca036-istio-token\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.943152 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.943019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-workload-certs\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.943152 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.943045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zzc5\" (UniqueName: \"kubernetes.io/projected/e28c4038-41cf-46fd-91e3-410d097ca036-kube-api-access-8zzc5\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.943152 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.943083    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-credential-socket\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.943152 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.943121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-istio-envoy\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.943152 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.943151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-workload-socket\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.943476 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.943216    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e28c4038-41cf-46fd-91e3-410d097ca036-istiod-ca-cert\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.943476 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.943245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-istio-data\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.943590 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.943545    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-credential-socket\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.943671 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.943649    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-workload-certs\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.943774 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.943756    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-workload-socket\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.943823 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.943762    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-istio-data\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.944130 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.944107    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e28c4038-41cf-46fd-91e3-410d097ca036-istiod-ca-cert\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.945610 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.945556    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e28c4038-41cf-46fd-91e3-410d097ca036-istio-podinfo\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.945894 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.945877    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-istio-envoy\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.951525 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.951326    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zzc5\" (UniqueName: \"kubernetes.io/projected/e28c4038-41cf-46fd-91e3-410d097ca036-kube-api-access-8zzc5\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:08.953027 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:08.951860    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e28c4038-41cf-46fd-91e3-410d097ca036-istio-token\") pod \"e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") " pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:09.087059 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:09.086939    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:09.151529 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:09.151469    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:09.364471 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:09.364296    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"]
Sep 08 20:44:09.366766 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:44:09.366740    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28c4038_41cf_46fd_91e3_410d097ca036.slice/crio-f4db85e4f0354503fb5f3d063076abef7f9ab8c4d5cef793420c6f4487893f11 WatchSource:0}: Error finding container f4db85e4f0354503fb5f3d063076abef7f9ab8c4d5cef793420c6f4487893f11: Status 404 returned error can't find the container with id f4db85e4f0354503fb5f3d063076abef7f9ab8c4d5cef793420c6f4487893f11
Sep 08 20:44:09.369168 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:09.369132    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:44:09.369265 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:09.369201    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:44:09.369265 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:09.369229    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:44:10.278586 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:10.278545    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7" event={"ID":"e28c4038-41cf-46fd-91e3-410d097ca036","Type":"ContainerStarted","Data":"fc2c8c186b06c17034810a702a239bb0a50ced52c2c222b47980546e1203856f"}
Sep 08 20:44:10.279066 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:10.278605    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7" event={"ID":"e28c4038-41cf-46fd-91e3-410d097ca036","Type":"ContainerStarted","Data":"f4db85e4f0354503fb5f3d063076abef7f9ab8c4d5cef793420c6f4487893f11"}
Sep 08 20:44:10.332676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:10.332616    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7" podStartSLOduration=2.332594278 podStartE2EDuration="2.332594278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:44:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:44:10.332277674 +0000 UTC m=+1123.879527156" watchObservedRunningTime="2026-09-08 20:44:10.332594278 +0000 UTC m=+1123.879843759"
Sep 08 20:44:10.984668 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:10.984635    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:11.087926 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.087889    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:11.092886 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.092858    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:11.283358 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.283267    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:11.284212 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.284192    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:11.321780 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.321744    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:11.581797 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.581699    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"]
Sep 08 20:44:11.584218 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.584201    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.586822 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.586799    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-c9590b-openshift-default-dockercfg-f4cts\""
Sep 08 20:44:11.597834 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.597808    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"]
Sep 08 20:44:11.615799 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.615769    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:11.670121 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.670080    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqxfm\" (UniqueName: \"kubernetes.io/projected/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-kube-api-access-jqxfm\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.670121 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.670124    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istiod-ca-cert\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.670381 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.670167    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-data\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.670381 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.670237    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-credential-socket\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.670381 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.670277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-envoy\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.670381 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.670299    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-podinfo\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.670381 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.670333    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-token\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.670614 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.670425    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-workload-socket\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.670614 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.670489    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-workload-certs\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.771771 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.771728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-podinfo\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.771771 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.771773    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-token\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.772083 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.771810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-workload-socket\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.772083 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.771833    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-workload-certs\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.772083 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.771859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqxfm\" (UniqueName: \"kubernetes.io/projected/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-kube-api-access-jqxfm\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.772083 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.771884    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istiod-ca-cert\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.772083 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.771943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-data\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.772083 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.772017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-credential-socket\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.772083 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.772056    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-envoy\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.772419 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.772358    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-workload-certs\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.772419 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.772384    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-workload-socket\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.772507 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.772456    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-credential-socket\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.772592 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.772572    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-data\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.772859 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.772841    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istiod-ca-cert\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.774271 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.774239    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-envoy\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.774523 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.774503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-podinfo\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.779381 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.779355    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-token\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.779467 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.779429    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqxfm\" (UniqueName: \"kubernetes.io/projected/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-kube-api-access-jqxfm\") pod \"e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") " pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:11.884271 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.884243    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:11.895924 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:11.895900    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:12.169421 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:12.169394    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"]
Sep 08 20:44:12.171189 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:44:12.171163    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c4e36f7_8249_4ccf_8f70_3d608af0b5ef.slice/crio-2fa83f15241851fad51332e9331fa4a0f36002b52c16c5908ca134640fd48e6b WatchSource:0}: Error finding container 2fa83f15241851fad51332e9331fa4a0f36002b52c16c5908ca134640fd48e6b: Status 404 returned error can't find the container with id 2fa83f15241851fad51332e9331fa4a0f36002b52c16c5908ca134640fd48e6b
Sep 08 20:44:12.173610 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:12.173575    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:44:12.173689 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:12.173669    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:44:12.173734 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:12.173717    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:44:12.235553 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:12.235516    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:12.289871 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:12.289825    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv" event={"ID":"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef","Type":"ContainerStarted","Data":"096f5d0e47453fab5034b774df951f15fefb0456fd79c002d0d124717407fd17"}
Sep 08 20:44:12.290379 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:12.289896    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv" event={"ID":"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef","Type":"ContainerStarted","Data":"2fa83f15241851fad51332e9331fa4a0f36002b52c16c5908ca134640fd48e6b"}
Sep 08 20:44:12.309423 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:12.309317    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv" podStartSLOduration=1.309295053 podStartE2EDuration="1.309295053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:44:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:44:12.309032019 +0000 UTC m=+1125.856281499" watchObservedRunningTime="2026-09-08 20:44:12.309295053 +0000 UTC m=+1125.856544531"
Sep 08 20:44:12.314468 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:12.314432    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:12.896950 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:12.896921    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:13.547364 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:13.547329    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:13.650436 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:13.650359    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:13.834905 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:13.834822    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:13.901466 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:13.901437    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:14.004773 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:14.004739    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:14.092512 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:14.092432    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:14.206752 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:14.206715    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:14.300530 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:14.300501    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:14.302054 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:14.302029    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:14.308697 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:14.308678    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:15.594083 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.594048    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"]
Sep 08 20:44:15.596632 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.596614    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:44:15.599624 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.599601    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-c9590b-serving-cert\""
Sep 08 20:44:15.611016 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.610989    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"]
Sep 08 20:44:15.632913 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.632879    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:15.709417 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.709382    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27dwf\" (UniqueName: \"kubernetes.io/projected/6bc743e9-36df-4ade-a4f6-91d275f9a05e-kube-api-access-27dwf\") pod \"maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn\" (UID: \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:44:15.709584 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.709432    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bc743e9-36df-4ade-a4f6-91d275f9a05e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn\" (UID: \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:44:15.709584 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.709467    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bc743e9-36df-4ade-a4f6-91d275f9a05e-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn\" (UID: \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:44:15.709584 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.709496    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bc743e9-36df-4ade-a4f6-91d275f9a05e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn\" (UID: \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:44:15.733193 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.733161    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:15.810634 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.810586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27dwf\" (UniqueName: \"kubernetes.io/projected/6bc743e9-36df-4ade-a4f6-91d275f9a05e-kube-api-access-27dwf\") pod \"maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn\" (UID: \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:44:15.810817 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.810652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bc743e9-36df-4ade-a4f6-91d275f9a05e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn\" (UID: \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:44:15.810817 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.810690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bc743e9-36df-4ade-a4f6-91d275f9a05e-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn\" (UID: \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:44:15.810817 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.810720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bc743e9-36df-4ade-a4f6-91d275f9a05e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn\" (UID: \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:44:15.811480 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.811457    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bc743e9-36df-4ade-a4f6-91d275f9a05e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn\" (UID: \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:44:15.813427 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.813397    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bc743e9-36df-4ade-a4f6-91d275f9a05e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn\" (UID: \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:44:15.813512 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.813440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bc743e9-36df-4ade-a4f6-91d275f9a05e-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn\" (UID: \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:44:15.821564 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.821536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27dwf\" (UniqueName: \"kubernetes.io/projected/6bc743e9-36df-4ade-a4f6-91d275f9a05e-kube-api-access-27dwf\") pod \"maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn\" (UID: \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:44:15.852735 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.852662    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:15.909820 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:15.909787    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:44:16.077183 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:16.077154    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:16.177687 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:16.177654    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:16.181862 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:16.181839    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"]
Sep 08 20:44:16.183650 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:44:16.183620    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bc743e9_36df_4ade_a4f6_91d275f9a05e.slice/crio-a4b21c49e195b828e6b96038a122975348cdb0e70cb29508cc30b84e81f48694 WatchSource:0}: Error finding container a4b21c49e195b828e6b96038a122975348cdb0e70cb29508cc30b84e81f48694: Status 404 returned error can't find the container with id a4b21c49e195b828e6b96038a122975348cdb0e70cb29508cc30b84e81f48694
Sep 08 20:44:16.314010 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:16.313972    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn" event={"ID":"6bc743e9-36df-4ade-a4f6-91d275f9a05e","Type":"ContainerStarted","Data":"a4b21c49e195b828e6b96038a122975348cdb0e70cb29508cc30b84e81f48694"}
Sep 08 20:44:16.799525 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:16.799484    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:17.320397 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:17.320364    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn" event={"ID":"6bc743e9-36df-4ade-a4f6-91d275f9a05e","Type":"ContainerStarted","Data":"aec40e3399d8f33135227c408381063038efc7e70582ee2f40d4655c03c4fbd0"}
Sep 08 20:44:17.320586 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:17.320493    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:44:17.339446 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:17.339394    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn" podStartSLOduration=1.906589826 podStartE2EDuration="2.339379659s" totalImagesPullingTime="432.789833ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:44:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:44:17.337820453 +0000 UTC m=+1130.885069934" watchObservedRunningTime="2026-09-08 20:44:17.339379659 +0000 UTC m=+1130.886629141"
Sep 08 20:44:17.508196 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:17.508158    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:23.331458 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:23.331426    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:44:30.342184 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:30.342130    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 08 20:44:30.342587 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:30.342217    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv" podUID="0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 08 20:44:30.609523 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:44:30.609488    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@4-15708-0.service\": RecentStats: unable to find data in memory cache]"
Sep 08 20:44:31.408588 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:31.408554    2580 generic.go:356] "Generic (PLEG): container finished" podID="0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" containerID="096f5d0e47453fab5034b774df951f15fefb0456fd79c002d0d124717407fd17" exitCode=0
Sep 08 20:44:31.409033 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:31.408630    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv" event={"ID":"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef","Type":"ContainerDied","Data":"096f5d0e47453fab5034b774df951f15fefb0456fd79c002d0d124717407fd17"}
Sep 08 20:44:31.409093 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:31.409050    2580 scope.go:117] "RemoveContainer" containerID="096f5d0e47453fab5034b774df951f15fefb0456fd79c002d0d124717407fd17"
Sep 08 20:44:31.410286 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:31.410253    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:44:31.410403 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:31.410335    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:44:31.410403 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:31.410376    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:44:31.410493 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:31.410476    2580 generic.go:356] "Generic (PLEG): container finished" podID="e28c4038-41cf-46fd-91e3-410d097ca036" containerID="fc2c8c186b06c17034810a702a239bb0a50ced52c2c222b47980546e1203856f" exitCode=0
Sep 08 20:44:31.410536 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:31.410502    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7" event={"ID":"e28c4038-41cf-46fd-91e3-410d097ca036","Type":"ContainerDied","Data":"fc2c8c186b06c17034810a702a239bb0a50ced52c2c222b47980546e1203856f"}
Sep 08 20:44:31.410891 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:31.410831    2580 scope.go:117] "RemoveContainer" containerID="fc2c8c186b06c17034810a702a239bb0a50ced52c2c222b47980546e1203856f"
Sep 08 20:44:31.411819 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:31.411794    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:44:31.411926 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:31.411864    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:44:31.411926 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:31.411905    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:44:32.418182 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:32.418106    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7" event={"ID":"e28c4038-41cf-46fd-91e3-410d097ca036","Type":"ContainerStarted","Data":"ab0ec76415b5b45540071b930eff353509b299ff722e1a827e34a47b1cd9aaad"}
Sep 08 20:44:32.420195 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:32.420171    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv" event={"ID":"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef","Type":"ContainerStarted","Data":"a8cbabd9d7b464f9f11278fc34807b69d0a24dff4b45b0911adbc24e9c32a733"}
Sep 08 20:44:32.896525 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:32.896492    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:32.896850 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:32.896782    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 08 20:44:32.897011 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:32.896994    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv" podUID="0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 08 20:44:33.087870 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:33.087825    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:33.088178 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:33.088146    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 08 20:44:33.088382 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:33.088353    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7" podUID="e28c4038-41cf-46fd-91e3-410d097ca036" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 08 20:44:33.897261 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:33.897220    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 08 20:44:33.897688 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:33.897289    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv" podUID="0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 08 20:44:34.088097 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:34.087610    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 08 20:44:34.088097 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:34.087683    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7" podUID="e28c4038-41cf-46fd-91e3-410d097ca036" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 08 20:44:34.896477 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:34.896437    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 08 20:44:34.896655 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:34.896511    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv" podUID="0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 08 20:44:35.087497 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:35.087456    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 08 20:44:35.088092 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:35.088057    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7" podUID="e28c4038-41cf-46fd-91e3-410d097ca036" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 08 20:44:35.724998 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:35.724940    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:35.897168 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:35.897127    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 08 20:44:35.897343 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:35.897182    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv" podUID="0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 08 20:44:36.087836 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:36.087730    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 08 20:44:36.088440 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:36.088405    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7" podUID="e28c4038-41cf-46fd-91e3-410d097ca036" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 08 20:44:36.897040 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:36.897001    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 08 20:44:36.897237 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:36.897061    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv" podUID="0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 08 20:44:36.992294 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:36.992254    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:37.087547 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:37.087499    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 08 20:44:37.087770 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:37.087599    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7" podUID="e28c4038-41cf-46fd-91e3-410d097ca036" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 08 20:44:37.199442 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:37.199343    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:37.654214 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:37.654172    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:37.900308 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:37.900279    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:37.900563 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:37.900537    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:37.901297 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:37.901278    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:44:38.092365 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:38.092283    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:38.092566 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:38.092544    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:38.093159 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:38.093143    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:44:38.348464 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:38.348376    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:39.224516 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:39.224487    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:49.326337 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:49.326256    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:50.622494 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.622460    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:50.685621 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.685587    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"]
Sep 08 20:44:50.691149 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.691125    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.694201 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.694173    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-78dd9f\"/\"default-dockercfg-r8s7w\""
Sep 08 20:44:50.694374 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.694231    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-78dd9f\"/\"kube-root-ca.crt\""
Sep 08 20:44:50.694374 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.694317    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-78dd9f\"/\"openshift-service-ca.crt\""
Sep 08 20:44:50.694504 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.694403    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-78dd9f\"/\"auth-test-model-78dd9f-kserve-self-signed-certs\""
Sep 08 20:44:50.697917 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.697894    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"]
Sep 08 20:44:50.718011 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.717968    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:50.766847 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.766808    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz69t\" (UniqueName: \"kubernetes.io/projected/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-kube-api-access-sz69t\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.767044 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.766858    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-dshm\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.767044 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.766877    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-model-cache\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.767044 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.766942    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-tmp-dir\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.767044 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.767004    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-tls-certs\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.767181 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.767063    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-home\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.867912 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.867878    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-home\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.868130 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.867952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz69t\" (UniqueName: \"kubernetes.io/projected/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-kube-api-access-sz69t\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.868130 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.868032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-dshm\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.868130 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.868061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-model-cache\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.868130 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.868109    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-tmp-dir\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.868371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.868150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-tls-certs\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.868433 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.868365    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-home\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.868573 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.868546    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-model-cache\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.868715 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.868601    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-tmp-dir\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.870705 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.870676    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-dshm\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.870933 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.870912    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-tls-certs\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.885642 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.885606    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz69t\" (UniqueName: \"kubernetes.io/projected/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-kube-api-access-sz69t\") pod \"auth-test-model-78dd9f-kserve-fcddd5555-l9c6x\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:50.934268 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:50.934218    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:51.004604 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:51.004564    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:44:51.074782 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:51.074666    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:51.281570 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:51.281545    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"]
Sep 08 20:44:51.283388 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:44:51.283361    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8de184f_a6bc_4795_bba0_2e7ffbc1d6d9.slice/crio-c7ea5391c1b7a0fdb864b645f319c3a842411b9883fff6cb7c17442192219533 WatchSource:0}: Error finding container c7ea5391c1b7a0fdb864b645f319c3a842411b9883fff6cb7c17442192219533: Status 404 returned error can't find the container with id c7ea5391c1b7a0fdb864b645f319c3a842411b9883fff6cb7c17442192219533
Sep 08 20:44:51.524450 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:51.524416    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x" event={"ID":"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9","Type":"ContainerStarted","Data":"5f1391d7f227a73f0f21cb6f3102243216279f6a623bf0f4098c83e02068f944"}
Sep 08 20:44:51.524632 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:51.524464    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x" event={"ID":"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9","Type":"ContainerStarted","Data":"c7ea5391c1b7a0fdb864b645f319c3a842411b9883fff6cb7c17442192219533"}
Sep 08 20:44:51.548261 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:51.548162    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x" podStartSLOduration=1.548146037 podStartE2EDuration="1.548146037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:44:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:44:51.546677674 +0000 UTC m=+1165.093927156" watchObservedRunningTime="2026-09-08 20:44:51.548146037 +0000 UTC m=+1165.095395517"
Sep 08 20:44:53.739007 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:53.738943    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:53.830824 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:53.830790    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:54.961300 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:54.961265    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:55.065377 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:55.065342    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:55.461841 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:55.461810    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:55.566500 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:55.566470    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:44:55.666284 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:44:55.666246    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:45:00.162541 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:00.162505    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-74f9a1-29815005-qc86w"]
Sep 08 20:45:00.166482 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:00.166454    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-74f9a1-29815005-qc86w"
Sep 08 20:45:00.168725 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:00.168701    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2b2qs\""
Sep 08 20:45:00.175980 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:00.175752    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-74f9a1-29815005-qc86w"]
Sep 08 20:45:00.180132 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:00.180107    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-74f9a1-29815005-qc86w"
Sep 08 20:45:00.452346 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:00.452316    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-74f9a1-29815005-qc86w"]
Sep 08 20:45:00.454632 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:45:00.454603    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea2ab14b_44f2_4d43_9113_345584fd63f3.slice/crio-8ea5b7ffac39d2b1368e01fdebea20b5b4bdc159d9d0539c37f96a26c4442e18 WatchSource:0}: Error finding container 8ea5b7ffac39d2b1368e01fdebea20b5b4bdc159d9d0539c37f96a26c4442e18: Status 404 returned error can't find the container with id 8ea5b7ffac39d2b1368e01fdebea20b5b4bdc159d9d0539c37f96a26c4442e18
Sep 08 20:45:00.573250 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:00.573204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-74f9a1-29815005-qc86w" event={"ID":"ea2ab14b-44f2-4d43-9113-345584fd63f3","Type":"ContainerStarted","Data":"8ea5b7ffac39d2b1368e01fdebea20b5b4bdc159d9d0539c37f96a26c4442e18"}
Sep 08 20:45:01.005070 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:01.005019    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:45:01.005070 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:01.005079    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:45:01.017548 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:01.017515    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:45:01.589214 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:01.589185    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:45:04.598311 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:04.598216    2580 generic.go:356] "Generic (PLEG): container finished" podID="ea2ab14b-44f2-4d43-9113-345584fd63f3" containerID="9a08a8a9d6cdd86680a7bc8f6633da7f8c8b85a414ed3c7923d6dcd685442c67" exitCode=0
Sep 08 20:45:04.598706 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:04.598301    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-74f9a1-29815005-qc86w" event={"ID":"ea2ab14b-44f2-4d43-9113-345584fd63f3","Type":"ContainerDied","Data":"9a08a8a9d6cdd86680a7bc8f6633da7f8c8b85a414ed3c7923d6dcd685442c67"}
Sep 08 20:45:05.739620 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:05.739595    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-74f9a1-29815005-qc86w"
Sep 08 20:45:06.609865 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:06.609834    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-74f9a1-29815005-qc86w"
Sep 08 20:45:06.610061 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:06.609872    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-74f9a1-29815005-qc86w" event={"ID":"ea2ab14b-44f2-4d43-9113-345584fd63f3","Type":"ContainerDied","Data":"8ea5b7ffac39d2b1368e01fdebea20b5b4bdc159d9d0539c37f96a26c4442e18"}
Sep 08 20:45:06.610061 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:06.609918    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ea5b7ffac39d2b1368e01fdebea20b5b4bdc159d9d0539c37f96a26c4442e18"
Sep 08 20:45:18.750486 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:18.750442    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:45:22.617033 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:22.616989    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:45:33.777839 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.777801    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"]
Sep 08 20:45:33.778261 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.778237    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea2ab14b-44f2-4d43-9113-345584fd63f3" containerName="cleanup"
Sep 08 20:45:33.778261 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.778249    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea2ab14b-44f2-4d43-9113-345584fd63f3" containerName="cleanup"
Sep 08 20:45:33.778334 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.778322    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea2ab14b-44f2-4d43-9113-345584fd63f3" containerName="cleanup"
Sep 08 20:45:33.783859 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.783840    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:33.786325 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.786291    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e0dc0027\"/\"openshift-service-ca.crt\""
Sep 08 20:45:33.786952 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.786872    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e0dc0027\"/\"kube-root-ca.crt\""
Sep 08 20:45:33.786952 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.786909    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e0dc0027\"/\"default-dockercfg-bx7vx\""
Sep 08 20:45:33.787146 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.786996    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e0dc0027\"/\"e2e-lifecycle-model-e0dc0027-kserve-self-signed-certs\""
Sep 08 20:45:33.788006 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.787987    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"]
Sep 08 20:45:33.905413 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.905371    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-dshm\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:33.905599 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.905442    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-tmp-dir\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:33.905599 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.905460    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-model-cache\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:33.905599 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.905478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k597l\" (UniqueName: \"kubernetes.io/projected/7700281f-cecc-426e-bfd9-bf81a3a9614d-kube-api-access-k597l\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:33.905599 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.905499    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7700281f-cecc-426e-bfd9-bf81a3a9614d-tls-certs\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:33.905762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.905614    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-home\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:33.983144 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:33.983096    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:45:34.006263 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.006220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-dshm\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:34.006457 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.006305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-tmp-dir\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:34.006457 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.006332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-model-cache\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:34.006457 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.006360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k597l\" (UniqueName: \"kubernetes.io/projected/7700281f-cecc-426e-bfd9-bf81a3a9614d-kube-api-access-k597l\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:34.006457 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.006387    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7700281f-cecc-426e-bfd9-bf81a3a9614d-tls-certs\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:34.006677 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.006470    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-home\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:34.006788 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.006764    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-tmp-dir\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:34.006859 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.006825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-model-cache\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:34.006859 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.006848    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-home\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:34.008701 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.008681    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-dshm\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:34.008865 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.008851    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7700281f-cecc-426e-bfd9-bf81a3a9614d-tls-certs\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:34.014171 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.014154    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k597l\" (UniqueName: \"kubernetes.io/projected/7700281f-cecc-426e-bfd9-bf81a3a9614d-kube-api-access-k597l\") pod \"e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") " pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:34.095627 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.095532    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:34.373158 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.373126    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"]
Sep 08 20:45:34.374778 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:45:34.374749    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7700281f_cecc_426e_bfd9_bf81a3a9614d.slice/crio-41bb86c75d00ce4c37cd556fb3dab1bcf06a233fae28eecf3add35f43874fb9a WatchSource:0}: Error finding container 41bb86c75d00ce4c37cd556fb3dab1bcf06a233fae28eecf3add35f43874fb9a: Status 404 returned error can't find the container with id 41bb86c75d00ce4c37cd556fb3dab1bcf06a233fae28eecf3add35f43874fb9a
Sep 08 20:45:34.762004 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.761953    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp" event={"ID":"7700281f-cecc-426e-bfd9-bf81a3a9614d","Type":"ContainerStarted","Data":"cb787816877bde9425c8cdebd9873bcff8e66dca6f6f293df35d9e756ba65d75"}
Sep 08 20:45:34.762198 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.762015    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp" event={"ID":"7700281f-cecc-426e-bfd9-bf81a3a9614d","Type":"ContainerStarted","Data":"41bb86c75d00ce4c37cd556fb3dab1bcf06a233fae28eecf3add35f43874fb9a"}
Sep 08 20:45:34.780161 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:34.780098    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp" podStartSLOduration=1.7800777220000001 podStartE2EDuration="1.780077722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:45:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:45:34.779866727 +0000 UTC m=+1208.327116209" watchObservedRunningTime="2026-09-08 20:45:34.780077722 +0000 UTC m=+1208.327327215"
Sep 08 20:45:44.096133 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:44.096088    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:44.096535 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:44.096154    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:44.109287 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:44.109257    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:44.826713 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:44.826684    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:45:48.188387 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.188354    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"]
Sep 08 20:45:48.193319 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.193292    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.197150 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.196923    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-c9590b\"/\"auth-test-model-c9590b-kserve-self-signed-certs\""
Sep 08 20:45:48.198497 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.197483    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-c9590b\"/\"kube-root-ca.crt\""
Sep 08 20:45:48.198497 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.197623    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-c9590b\"/\"openshift-service-ca.crt\""
Sep 08 20:45:48.198497 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.197740    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-c9590b\"/\"default-dockercfg-29kxk\""
Sep 08 20:45:48.204498 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.204470    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"]
Sep 08 20:45:48.226463 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.226429    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:45:48.356498 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.356443    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gp7w\" (UniqueName: \"kubernetes.io/projected/9bca515d-bc93-4526-9ddc-4643700d953d-kube-api-access-6gp7w\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.356988 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.356937    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-tmp-dir\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.357246 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.357227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-model-cache\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.357420 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.357404    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bca515d-bc93-4526-9ddc-4643700d953d-tls-certs\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.357570 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.357555    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-home\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.357717 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.357703    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-dshm\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.458743 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.458649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-model-cache\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.458743 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.458686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bca515d-bc93-4526-9ddc-4643700d953d-tls-certs\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.458743 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.458704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-home\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.458743 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.458726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-dshm\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.459089 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.458784    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gp7w\" (UniqueName: \"kubernetes.io/projected/9bca515d-bc93-4526-9ddc-4643700d953d-kube-api-access-6gp7w\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.459089 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.458859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-tmp-dir\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.459213 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.459177    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-model-cache\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.459275 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.459211    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-home\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.459365 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.459337    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-tmp-dir\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.461202 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.461177    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-dshm\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.461568 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.461549    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bca515d-bc93-4526-9ddc-4643700d953d-tls-certs\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.467754 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.467729    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gp7w\" (UniqueName: \"kubernetes.io/projected/9bca515d-bc93-4526-9ddc-4643700d953d-kube-api-access-6gp7w\") pod \"auth-test-model-c9590b-kserve-7945f7cb59-s2kb4\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.506438 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.506399    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:48.781747 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.781695    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"]
Sep 08 20:45:48.784428 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:45:48.784397    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9bca515d_bc93_4526_9ddc_4643700d953d.slice/crio-8be2b7ed80bb96dbb55a540b823edca7e48408ccbd40151d56ff2a32ecce15ab WatchSource:0}: Error finding container 8be2b7ed80bb96dbb55a540b823edca7e48408ccbd40151d56ff2a32ecce15ab: Status 404 returned error can't find the container with id 8be2b7ed80bb96dbb55a540b823edca7e48408ccbd40151d56ff2a32ecce15ab
Sep 08 20:45:48.837747 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:48.837713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4" event={"ID":"9bca515d-bc93-4526-9ddc-4643700d953d","Type":"ContainerStarted","Data":"8be2b7ed80bb96dbb55a540b823edca7e48408ccbd40151d56ff2a32ecce15ab"}
Sep 08 20:45:49.844766 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:49.844731    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4" event={"ID":"9bca515d-bc93-4526-9ddc-4643700d953d","Type":"ContainerStarted","Data":"24b568c93b52396ada21698f3e96687a6cf0f07b83a6c8e4a01071aab925ca8d"}
Sep 08 20:45:49.861682 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:49.861627    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4" podStartSLOduration=1.861611481 podStartE2EDuration="1.861611481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:45:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:45:49.861196624 +0000 UTC m=+1223.408446107" watchObservedRunningTime="2026-09-08 20:45:49.861611481 +0000 UTC m=+1223.408860961"
Sep 08 20:45:58.507028 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:58.506979    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:58.507500 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:58.507056    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:58.519351 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:58.519316    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:45:58.903277 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:45:58.903239    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:46:13.960173 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:13.960140    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:46:29.345974 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:29.345877    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:46:31.894229 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:31.894192    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:46:32.825249 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:32.825220    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:46:37.813169 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:37.813127    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"]
Sep 08 20:46:37.909341 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:37.909305    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:46:44.784384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:44.784345    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:46:45.993297 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:45.993266    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:46:52.833744 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:52.833703    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp" podUID="7700281f-cecc-426e-bfd9-bf81a3a9614d" containerName="main" containerID="cri-o://cb787816877bde9425c8cdebd9873bcff8e66dca6f6f293df35d9e756ba65d75" gracePeriod=45
Sep 08 20:46:53.086312 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.086248    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:46:53.172388 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.172358    2580 generic.go:356] "Generic (PLEG): container finished" podID="7700281f-cecc-426e-bfd9-bf81a3a9614d" containerID="cb787816877bde9425c8cdebd9873bcff8e66dca6f6f293df35d9e756ba65d75" exitCode=0
Sep 08 20:46:53.172558 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.172406    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"
Sep 08 20:46:53.172558 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.172437    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp" event={"ID":"7700281f-cecc-426e-bfd9-bf81a3a9614d","Type":"ContainerDied","Data":"cb787816877bde9425c8cdebd9873bcff8e66dca6f6f293df35d9e756ba65d75"}
Sep 08 20:46:53.172558 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.172487    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp" event={"ID":"7700281f-cecc-426e-bfd9-bf81a3a9614d","Type":"ContainerDied","Data":"41bb86c75d00ce4c37cd556fb3dab1bcf06a233fae28eecf3add35f43874fb9a"}
Sep 08 20:46:53.172558 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.172510    2580 scope.go:117] "RemoveContainer" containerID="cb787816877bde9425c8cdebd9873bcff8e66dca6f6f293df35d9e756ba65d75"
Sep 08 20:46:53.181160 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.181139    2580 scope.go:117] "RemoveContainer" containerID="cb787816877bde9425c8cdebd9873bcff8e66dca6f6f293df35d9e756ba65d75"
Sep 08 20:46:53.181439 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:46:53.181420    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb787816877bde9425c8cdebd9873bcff8e66dca6f6f293df35d9e756ba65d75\": container with ID starting with cb787816877bde9425c8cdebd9873bcff8e66dca6f6f293df35d9e756ba65d75 not found: ID does not exist" containerID="cb787816877bde9425c8cdebd9873bcff8e66dca6f6f293df35d9e756ba65d75"
Sep 08 20:46:53.181518 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.181446    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb787816877bde9425c8cdebd9873bcff8e66dca6f6f293df35d9e756ba65d75"} err="failed to get container status \"cb787816877bde9425c8cdebd9873bcff8e66dca6f6f293df35d9e756ba65d75\": rpc error: code = NotFound desc = could not find container \"cb787816877bde9425c8cdebd9873bcff8e66dca6f6f293df35d9e756ba65d75\": container with ID starting with cb787816877bde9425c8cdebd9873bcff8e66dca6f6f293df35d9e756ba65d75 not found: ID does not exist"
Sep 08 20:46:53.206216 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.206181    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k597l\" (UniqueName: \"kubernetes.io/projected/7700281f-cecc-426e-bfd9-bf81a3a9614d-kube-api-access-k597l\") pod \"7700281f-cecc-426e-bfd9-bf81a3a9614d\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") "
Sep 08 20:46:53.206362 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.206316    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7700281f-cecc-426e-bfd9-bf81a3a9614d-tls-certs\") pod \"7700281f-cecc-426e-bfd9-bf81a3a9614d\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") "
Sep 08 20:46:53.206362 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.206343    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-home\") pod \"7700281f-cecc-426e-bfd9-bf81a3a9614d\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") "
Sep 08 20:46:53.206476 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.206362    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-dshm\") pod \"7700281f-cecc-426e-bfd9-bf81a3a9614d\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") "
Sep 08 20:46:53.206476 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.206385    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-model-cache\") pod \"7700281f-cecc-426e-bfd9-bf81a3a9614d\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") "
Sep 08 20:46:53.206476 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.206421    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-tmp-dir\") pod \"7700281f-cecc-426e-bfd9-bf81a3a9614d\" (UID: \"7700281f-cecc-426e-bfd9-bf81a3a9614d\") "
Sep 08 20:46:53.206904 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.206626    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-home" (OuterVolumeSpecName: "home") pod "7700281f-cecc-426e-bfd9-bf81a3a9614d" (UID: "7700281f-cecc-426e-bfd9-bf81a3a9614d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:46:53.206904 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.206714    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-model-cache" (OuterVolumeSpecName: "model-cache") pod "7700281f-cecc-426e-bfd9-bf81a3a9614d" (UID: "7700281f-cecc-426e-bfd9-bf81a3a9614d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:46:53.206904 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.206808    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-home\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:46:53.206904 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.206827    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-model-cache\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:46:53.206904 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.206865    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7700281f-cecc-426e-bfd9-bf81a3a9614d" (UID: "7700281f-cecc-426e-bfd9-bf81a3a9614d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:46:53.208556 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.208533    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-dshm" (OuterVolumeSpecName: "dshm") pod "7700281f-cecc-426e-bfd9-bf81a3a9614d" (UID: "7700281f-cecc-426e-bfd9-bf81a3a9614d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:46:53.208983 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.208927    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7700281f-cecc-426e-bfd9-bf81a3a9614d-kube-api-access-k597l" (OuterVolumeSpecName: "kube-api-access-k597l") pod "7700281f-cecc-426e-bfd9-bf81a3a9614d" (UID: "7700281f-cecc-426e-bfd9-bf81a3a9614d"). InnerVolumeSpecName "kube-api-access-k597l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:46:53.209138 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.209115    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7700281f-cecc-426e-bfd9-bf81a3a9614d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7700281f-cecc-426e-bfd9-bf81a3a9614d" (UID: "7700281f-cecc-426e-bfd9-bf81a3a9614d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:46:53.308049 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.308016    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7700281f-cecc-426e-bfd9-bf81a3a9614d-tls-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:46:53.308049 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.308043    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-dshm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:46:53.308049 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.308053    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7700281f-cecc-426e-bfd9-bf81a3a9614d-tmp-dir\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:46:53.308312 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.308061    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k597l\" (UniqueName: \"kubernetes.io/projected/7700281f-cecc-426e-bfd9-bf81a3a9614d-kube-api-access-k597l\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:46:53.494919 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.494882    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"]
Sep 08 20:46:53.496907 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:53.496885    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-e0dc0027/e2e-lifecycle-model-e0dc0027-kserve-6df97fc77f-wgfpp"]
Sep 08 20:46:54.925419 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:46:54.925386    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7700281f-cecc-426e-bfd9-bf81a3a9614d" path="/var/lib/kubelet/pods/7700281f-cecc-426e-bfd9-bf81a3a9614d/volumes"
Sep 08 20:47:09.214100 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:09.214063    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:09.270072 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:09.270035    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:16.380214 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:16.380177    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:23.878707 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:23.878669    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:24.476614 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:24.476579    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:29.966857 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:29.966817    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:35.852799 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:35.852764    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:35.918092 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:35.918038    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:36.106058 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:36.105947    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:37.367199 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:37.367166    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:37.463653 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:37.463618    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:37.564365 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:37.564335    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:38.261690 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:38.261657    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:38.380887 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:38.380851    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:38.536450 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:38.536365    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:39.763772 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:39.763736    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:45.641056 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:45.641022    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:45.744522 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:45.744479    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:46.324239 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:46.324205    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:52.304638 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:52.304600    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:52.405802 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:52.405765    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:52.505338 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:52.505302    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:56.786891 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:56.786857    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"]
Sep 08 20:47:56.805811 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:56.805775    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:47:57.883226 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:57.883182    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"]
Sep 08 20:47:57.905315 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:47:57.905285    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:03.619055 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:03.619021    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:03.699886 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:03.699848    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:11.095387 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.095342    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"]
Sep 08 20:48:11.096018 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.095997    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7700281f-cecc-426e-bfd9-bf81a3a9614d" containerName="main"
Sep 08 20:48:11.096106 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.096019    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7700281f-cecc-426e-bfd9-bf81a3a9614d" containerName="main"
Sep 08 20:48:11.096166 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.096146    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7700281f-cecc-426e-bfd9-bf81a3a9614d" containerName="main"
Sep 08 20:48:11.099293 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.099272    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.101494 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.101466    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-78dd9f\"/\"sub-test-model-78dd9f-kserve-self-signed-certs\""
Sep 08 20:48:11.110801 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.110770    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"]
Sep 08 20:48:11.121447 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.121417    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:11.215945 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.215898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-dshm\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.215945 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.215942    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66jnm\" (UniqueName: \"kubernetes.io/projected/c5c4f240-9922-4f58-bac1-1026f9598a1d-kube-api-access-66jnm\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.216195 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.215994    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5c4f240-9922-4f58-bac1-1026f9598a1d-tls-certs\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.216195 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.216068    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-home\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.216195 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.216095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-model-cache\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.216195 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.216135    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-tmp-dir\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.317368 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.317328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-dshm\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.317368 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.317368    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66jnm\" (UniqueName: \"kubernetes.io/projected/c5c4f240-9922-4f58-bac1-1026f9598a1d-kube-api-access-66jnm\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.317604 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.317395    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5c4f240-9922-4f58-bac1-1026f9598a1d-tls-certs\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.317604 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.317431    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-home\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.317604 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.317447    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-model-cache\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.317765 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.317607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-tmp-dir\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.317874 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.317847    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-model-cache\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.317874 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.317862    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-home\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.318048 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.317923    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-tmp-dir\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.319765 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.319735    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-dshm\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.319907 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.319889    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5c4f240-9922-4f58-bac1-1026f9598a1d-tls-certs\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.325283 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.325260    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66jnm\" (UniqueName: \"kubernetes.io/projected/c5c4f240-9922-4f58-bac1-1026f9598a1d-kube-api-access-66jnm\") pod \"sub-test-model-78dd9f-kserve-7b79df9575-c9tww\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.410202 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.410158    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:11.684552 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.684526    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"]
Sep 08 20:48:11.686711 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:48:11.686677    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5c4f240_9922_4f58_bac1_1026f9598a1d.slice/crio-7e665999e507dc5e5e057c5667e9da0279346192991752a15e9dd2d4fc057f8b WatchSource:0}: Error finding container 7e665999e507dc5e5e057c5667e9da0279346192991752a15e9dd2d4fc057f8b: Status 404 returned error can't find the container with id 7e665999e507dc5e5e057c5667e9da0279346192991752a15e9dd2d4fc057f8b
Sep 08 20:48:11.807705 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:11.807638    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x" podUID="f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9" containerName="main" containerID="cri-o://5f1391d7f227a73f0f21cb6f3102243216279f6a623bf0f4098c83e02068f944" gracePeriod=45
Sep 08 20:48:12.045135 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.045107    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:48:12.227770 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.227656    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-home\") pod \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") "
Sep 08 20:48:12.227770 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.227747    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-model-cache\") pod \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") "
Sep 08 20:48:12.228334 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.227782    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-tls-certs\") pod \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") "
Sep 08 20:48:12.228334 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.227808    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-tmp-dir\") pod \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") "
Sep 08 20:48:12.228334 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.227846    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-dshm\") pod \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") "
Sep 08 20:48:12.228334 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.227878    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sz69t\" (UniqueName: \"kubernetes.io/projected/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-kube-api-access-sz69t\") pod \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\" (UID: \"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9\") "
Sep 08 20:48:12.228334 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.227953    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-home" (OuterVolumeSpecName: "home") pod "f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9" (UID: "f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:48:12.228334 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.228117    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9" (UID: "f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:48:12.228334 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.228159    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9" (UID: "f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:48:12.228334 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.228279    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-tmp-dir\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:48:12.228334 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.228295    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-home\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:48:12.228334 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.228304    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-model-cache\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:48:12.230124 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.230103    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-dshm" (OuterVolumeSpecName: "dshm") pod "f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9" (UID: "f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:48:12.230124 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.230111    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9" (UID: "f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:48:12.230253 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.230165    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-kube-api-access-sz69t" (OuterVolumeSpecName: "kube-api-access-sz69t") pod "f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9" (UID: "f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9"). InnerVolumeSpecName "kube-api-access-sz69t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:48:12.329489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.329459    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-dshm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:48:12.329489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.329492    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sz69t\" (UniqueName: \"kubernetes.io/projected/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-kube-api-access-sz69t\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:48:12.329619 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.329503    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9-tls-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:48:12.583361 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.583272    2580 generic.go:356] "Generic (PLEG): container finished" podID="f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9" containerID="5f1391d7f227a73f0f21cb6f3102243216279f6a623bf0f4098c83e02068f944" exitCode=0
Sep 08 20:48:12.583361 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.583312    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"
Sep 08 20:48:12.583574 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.583355    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x" event={"ID":"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9","Type":"ContainerDied","Data":"5f1391d7f227a73f0f21cb6f3102243216279f6a623bf0f4098c83e02068f944"}
Sep 08 20:48:12.583574 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.583395    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x" event={"ID":"f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9","Type":"ContainerDied","Data":"c7ea5391c1b7a0fdb864b645f319c3a842411b9883fff6cb7c17442192219533"}
Sep 08 20:48:12.583574 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.583412    2580 scope.go:117] "RemoveContainer" containerID="5f1391d7f227a73f0f21cb6f3102243216279f6a623bf0f4098c83e02068f944"
Sep 08 20:48:12.585487 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.585454    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww" event={"ID":"c5c4f240-9922-4f58-bac1-1026f9598a1d","Type":"ContainerStarted","Data":"abbefa351ecc3179aa7f599181c6179b77537692d85fe6627a634f12a303325a"}
Sep 08 20:48:12.585605 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.585503    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww" event={"ID":"c5c4f240-9922-4f58-bac1-1026f9598a1d","Type":"ContainerStarted","Data":"7e665999e507dc5e5e057c5667e9da0279346192991752a15e9dd2d4fc057f8b"}
Sep 08 20:48:12.592890 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.592873    2580 scope.go:117] "RemoveContainer" containerID="5f1391d7f227a73f0f21cb6f3102243216279f6a623bf0f4098c83e02068f944"
Sep 08 20:48:12.593215 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:48:12.593193    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f1391d7f227a73f0f21cb6f3102243216279f6a623bf0f4098c83e02068f944\": container with ID starting with 5f1391d7f227a73f0f21cb6f3102243216279f6a623bf0f4098c83e02068f944 not found: ID does not exist" containerID="5f1391d7f227a73f0f21cb6f3102243216279f6a623bf0f4098c83e02068f944"
Sep 08 20:48:12.593296 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.593223    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f1391d7f227a73f0f21cb6f3102243216279f6a623bf0f4098c83e02068f944"} err="failed to get container status \"5f1391d7f227a73f0f21cb6f3102243216279f6a623bf0f4098c83e02068f944\": rpc error: code = NotFound desc = could not find container \"5f1391d7f227a73f0f21cb6f3102243216279f6a623bf0f4098c83e02068f944\": container with ID starting with 5f1391d7f227a73f0f21cb6f3102243216279f6a623bf0f4098c83e02068f944 not found: ID does not exist"
Sep 08 20:48:12.602660 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.602620    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww" podStartSLOduration=1.602607205 podStartE2EDuration="1.602607205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:48:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:48:12.602086544 +0000 UTC m=+1366.149336025" watchObservedRunningTime="2026-09-08 20:48:12.602607205 +0000 UTC m=+1366.149856685"
Sep 08 20:48:12.614611 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.614574    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"]
Sep 08 20:48:12.616558 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.616527    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/auth-test-model-78dd9f-kserve-fcddd5555-l9c6x"]
Sep 08 20:48:12.903901 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.903842    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4" podUID="9bca515d-bc93-4526-9ddc-4643700d953d" containerName="main" containerID="cri-o://24b568c93b52396ada21698f3e96687a6cf0f07b83a6c8e4a01071aab925ca8d" gracePeriod=45
Sep 08 20:48:12.926312 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:12.926282    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9" path="/var/lib/kubelet/pods/f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9/volumes"
Sep 08 20:48:13.156628 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.156558    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:48:13.238252 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.238194    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-model-cache\") pod \"9bca515d-bc93-4526-9ddc-4643700d953d\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") "
Sep 08 20:48:13.238718 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.238328    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-tmp-dir\") pod \"9bca515d-bc93-4526-9ddc-4643700d953d\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") "
Sep 08 20:48:13.238718 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.238359    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gp7w\" (UniqueName: \"kubernetes.io/projected/9bca515d-bc93-4526-9ddc-4643700d953d-kube-api-access-6gp7w\") pod \"9bca515d-bc93-4526-9ddc-4643700d953d\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") "
Sep 08 20:48:13.238718 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.238379    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bca515d-bc93-4526-9ddc-4643700d953d-tls-certs\") pod \"9bca515d-bc93-4526-9ddc-4643700d953d\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") "
Sep 08 20:48:13.238718 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.238398    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-home\") pod \"9bca515d-bc93-4526-9ddc-4643700d953d\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") "
Sep 08 20:48:13.238718 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.238414    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-dshm\") pod \"9bca515d-bc93-4526-9ddc-4643700d953d\" (UID: \"9bca515d-bc93-4526-9ddc-4643700d953d\") "
Sep 08 20:48:13.238718 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.238512    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-model-cache" (OuterVolumeSpecName: "model-cache") pod "9bca515d-bc93-4526-9ddc-4643700d953d" (UID: "9bca515d-bc93-4526-9ddc-4643700d953d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:48:13.238718 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.238634    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9bca515d-bc93-4526-9ddc-4643700d953d" (UID: "9bca515d-bc93-4526-9ddc-4643700d953d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:48:13.238718 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.238651    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-home" (OuterVolumeSpecName: "home") pod "9bca515d-bc93-4526-9ddc-4643700d953d" (UID: "9bca515d-bc93-4526-9ddc-4643700d953d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:48:13.239092 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.238761    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-tmp-dir\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:48:13.239092 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.238779    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-home\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:48:13.239092 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.238789    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-model-cache\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:48:13.240707 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.240673    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-dshm" (OuterVolumeSpecName: "dshm") pod "9bca515d-bc93-4526-9ddc-4643700d953d" (UID: "9bca515d-bc93-4526-9ddc-4643700d953d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:48:13.240813 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.240769    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bca515d-bc93-4526-9ddc-4643700d953d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9bca515d-bc93-4526-9ddc-4643700d953d" (UID: "9bca515d-bc93-4526-9ddc-4643700d953d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:48:13.240813 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.240782    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bca515d-bc93-4526-9ddc-4643700d953d-kube-api-access-6gp7w" (OuterVolumeSpecName: "kube-api-access-6gp7w") pod "9bca515d-bc93-4526-9ddc-4643700d953d" (UID: "9bca515d-bc93-4526-9ddc-4643700d953d"). InnerVolumeSpecName "kube-api-access-6gp7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:48:13.340348 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.340313    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gp7w\" (UniqueName: \"kubernetes.io/projected/9bca515d-bc93-4526-9ddc-4643700d953d-kube-api-access-6gp7w\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:48:13.340348 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.340343    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bca515d-bc93-4526-9ddc-4643700d953d-tls-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:48:13.340348 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.340352    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bca515d-bc93-4526-9ddc-4643700d953d-dshm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:48:13.595578 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.595484    2580 generic.go:356] "Generic (PLEG): container finished" podID="9bca515d-bc93-4526-9ddc-4643700d953d" containerID="24b568c93b52396ada21698f3e96687a6cf0f07b83a6c8e4a01071aab925ca8d" exitCode=0
Sep 08 20:48:13.595578 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.595565    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4" event={"ID":"9bca515d-bc93-4526-9ddc-4643700d953d","Type":"ContainerDied","Data":"24b568c93b52396ada21698f3e96687a6cf0f07b83a6c8e4a01071aab925ca8d"}
Sep 08 20:48:13.595821 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.595604    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"
Sep 08 20:48:13.595821 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.595620    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4" event={"ID":"9bca515d-bc93-4526-9ddc-4643700d953d","Type":"ContainerDied","Data":"8be2b7ed80bb96dbb55a540b823edca7e48408ccbd40151d56ff2a32ecce15ab"}
Sep 08 20:48:13.595821 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.595641    2580 scope.go:117] "RemoveContainer" containerID="24b568c93b52396ada21698f3e96687a6cf0f07b83a6c8e4a01071aab925ca8d"
Sep 08 20:48:13.605887 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.605869    2580 scope.go:117] "RemoveContainer" containerID="24b568c93b52396ada21698f3e96687a6cf0f07b83a6c8e4a01071aab925ca8d"
Sep 08 20:48:13.606252 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:48:13.606232    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24b568c93b52396ada21698f3e96687a6cf0f07b83a6c8e4a01071aab925ca8d\": container with ID starting with 24b568c93b52396ada21698f3e96687a6cf0f07b83a6c8e4a01071aab925ca8d not found: ID does not exist" containerID="24b568c93b52396ada21698f3e96687a6cf0f07b83a6c8e4a01071aab925ca8d"
Sep 08 20:48:13.606356 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.606259    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24b568c93b52396ada21698f3e96687a6cf0f07b83a6c8e4a01071aab925ca8d"} err="failed to get container status \"24b568c93b52396ada21698f3e96687a6cf0f07b83a6c8e4a01071aab925ca8d\": rpc error: code = NotFound desc = could not find container \"24b568c93b52396ada21698f3e96687a6cf0f07b83a6c8e4a01071aab925ca8d\": container with ID starting with 24b568c93b52396ada21698f3e96687a6cf0f07b83a6c8e4a01071aab925ca8d not found: ID does not exist"
Sep 08 20:48:13.617870 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.617839    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"]
Sep 08 20:48:13.621875 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:13.621839    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/auth-test-model-c9590b-kserve-7945f7cb59-s2kb4"]
Sep 08 20:48:14.927440 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:14.927403    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bca515d-bc93-4526-9ddc-4643700d953d" path="/var/lib/kubelet/pods/9bca515d-bc93-4526-9ddc-4643700d953d/volumes"
Sep 08 20:48:20.757300 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:20.757253    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:20.858898 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:20.858840    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:21.411211 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:21.411176    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:21.411403 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:21.411251    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:21.423676 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:21.423647    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:21.651487 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:21.651455    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:48:24.203003 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:24.202968    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:29.914163 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:29.914125    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:30.014417 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.014380    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:30.304842 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.304753    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:30.603507 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.603422    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"]
Sep 08 20:48:30.603881 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.603867    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9" containerName="main"
Sep 08 20:48:30.603930 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.603883    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9" containerName="main"
Sep 08 20:48:30.603930 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.603892    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bca515d-bc93-4526-9ddc-4643700d953d" containerName="main"
Sep 08 20:48:30.603930 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.603898    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bca515d-bc93-4526-9ddc-4643700d953d" containerName="main"
Sep 08 20:48:30.604045 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.603985    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bca515d-bc93-4526-9ddc-4643700d953d" containerName="main"
Sep 08 20:48:30.604045 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.603995    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8de184f-a6bc-4795-bba0-2e7ffbc1d6d9" containerName="main"
Sep 08 20:48:30.608356 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.608338    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.612153 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.612134    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ce2cc8-openshift-default-dockercfg-mrp8t\""
Sep 08 20:48:30.619305 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.619281    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"]
Sep 08 20:48:30.632112 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.632067    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:30.725098 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.725064    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istiod-ca-cert\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.725322 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.725132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-token\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.725322 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.725213    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-workload-certs\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.725322 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.725276    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-workload-socket\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.725322 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.725301    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-envoy\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.725533 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.725342    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-podinfo\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.725533 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.725369    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-data\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.725533 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.725414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-credential-socket\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.725533 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.725454    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddq2r\" (UniqueName: \"kubernetes.io/projected/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-kube-api-access-ddq2r\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.826672 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.826633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istiod-ca-cert\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.826847 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.826698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-token\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.826847 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.826728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-workload-certs\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.826847 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.826760    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-workload-socket\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.826847 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.826776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-envoy\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.827112 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.826846    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-podinfo\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.827112 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.826880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-data\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.827112 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.826928    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-credential-socket\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.827112 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.827000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddq2r\" (UniqueName: \"kubernetes.io/projected/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-kube-api-access-ddq2r\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.827317 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.827208    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-workload-socket\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.827317 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.827285    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-workload-certs\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.827433 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.827372    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-data\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.827499 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.827469    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istiod-ca-cert\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.827499 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.827493    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-credential-socket\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.829133 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.829106    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-envoy\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.829395 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.829372    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-podinfo\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.835599 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.835570    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-token\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.835685 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.835656    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddq2r\" (UniqueName: \"kubernetes.io/projected/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-kube-api-access-ddq2r\") pod \"e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") " pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.918897 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.918865    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:30.994158 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:30.994126    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:31.194347 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:31.194321    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"]
Sep 08 20:48:31.196276 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:48:31.196246    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07a0bd22_9061_4fcd_9e3f_0e4aa48af090.slice/crio-5f2d68858c8103eefb52d944917a45eb83df81eb9c7451d492b1c025ed75bd0e WatchSource:0}: Error finding container 5f2d68858c8103eefb52d944917a45eb83df81eb9c7451d492b1c025ed75bd0e: Status 404 returned error can't find the container with id 5f2d68858c8103eefb52d944917a45eb83df81eb9c7451d492b1c025ed75bd0e
Sep 08 20:48:31.198677 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:31.198645    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:48:31.198747 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:31.198713    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:48:31.198747 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:31.198741    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:48:31.692337 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:31.692300    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6" event={"ID":"07a0bd22-9061-4fcd-9e3f-0e4aa48af090","Type":"ContainerStarted","Data":"8b547531491d818915a1ae298d01140dbf37986bab0aaeeb95d287677ef27f69"}
Sep 08 20:48:31.692519 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:31.692354    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6" event={"ID":"07a0bd22-9061-4fcd-9e3f-0e4aa48af090","Type":"ContainerStarted","Data":"5f2d68858c8103eefb52d944917a45eb83df81eb9c7451d492b1c025ed75bd0e"}
Sep 08 20:48:31.714549 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:31.714495    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6" podStartSLOduration=1.714479101 podStartE2EDuration="1.714479101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:48:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:48:31.713255514 +0000 UTC m=+1385.260505020" watchObservedRunningTime="2026-09-08 20:48:31.714479101 +0000 UTC m=+1385.261728582"
Sep 08 20:48:31.919194 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:31.919155    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:32.926188 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:32.926157    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:33.082573 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.082537    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"]
Sep 08 20:48:33.086074 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.086047    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.088362 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.088339    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-c9590b\"/\"default-dockercfg-29kxk\""
Sep 08 20:48:33.088455 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.088362    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-c9590b\"/\"openshift-service-ca.crt\""
Sep 08 20:48:33.088455 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.088410    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-c9590b\"/\"sub-test-model-c9590b-kserve-self-signed-certs\""
Sep 08 20:48:33.088455 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.088344    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-c9590b\"/\"kube-root-ca.crt\""
Sep 08 20:48:33.094804 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.094778    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"]
Sep 08 20:48:33.252479 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.252392    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-tmp-dir\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.252479 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.252465    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-dshm\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.252666 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.252514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mflb\" (UniqueName: \"kubernetes.io/projected/41d68009-87dd-42bf-9b29-49f6cf4da23c-kube-api-access-5mflb\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.252666 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.252572    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41d68009-87dd-42bf-9b29-49f6cf4da23c-tls-certs\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.252666 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.252616    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-model-cache\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.252762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.252670    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-home\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.293472 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.293437    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:33.353188 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.353149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-dshm\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.353353 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.353210    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mflb\" (UniqueName: \"kubernetes.io/projected/41d68009-87dd-42bf-9b29-49f6cf4da23c-kube-api-access-5mflb\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.353353 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.353250    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41d68009-87dd-42bf-9b29-49f6cf4da23c-tls-certs\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.353353 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.353291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-model-cache\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.353551 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.353355    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-home\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.353551 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.353394    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-tmp-dir\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.353733 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.353700    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-model-cache\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.353869 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.353751    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-home\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.353869 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.353772    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-tmp-dir\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.355559 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.355537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-dshm\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.355852 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.355831    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41d68009-87dd-42bf-9b29-49f6cf4da23c-tls-certs\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.361342 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.361320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mflb\" (UniqueName: \"kubernetes.io/projected/41d68009-87dd-42bf-9b29-49f6cf4da23c-kube-api-access-5mflb\") pod \"sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.397596 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.397561    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:33.665356 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.665313    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:33.678799 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.678775    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"]
Sep 08 20:48:33.680415 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:48:33.680388    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41d68009_87dd_42bf_9b29_49f6cf4da23c.slice/crio-c244fa1b7f538061cda43ace76bd190b703655b4078d87b1b727f5756e890ad2 WatchSource:0}: Error finding container c244fa1b7f538061cda43ace76bd190b703655b4078d87b1b727f5756e890ad2: Status 404 returned error can't find the container with id c244fa1b7f538061cda43ace76bd190b703655b4078d87b1b727f5756e890ad2
Sep 08 20:48:33.704353 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.704321    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9" event={"ID":"41d68009-87dd-42bf-9b29-49f6cf4da23c","Type":"ContainerStarted","Data":"c244fa1b7f538061cda43ace76bd190b703655b4078d87b1b727f5756e890ad2"}
Sep 08 20:48:33.704634 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.704616    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:33.705588 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.705571    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:33.786050 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:33.786017    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:34.710847 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:34.710808    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9" event={"ID":"41d68009-87dd-42bf-9b29-49f6cf4da23c","Type":"ContainerStarted","Data":"ce6cb87aae77d9f21d11c6eaaf517de9a50239af6fa7e4e695d3bfd5b9dfa097"}
Sep 08 20:48:34.728288 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:34.728223    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9" podStartSLOduration=1.728202491 podStartE2EDuration="1.728202491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:48:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:48:34.72711007 +0000 UTC m=+1388.274359554" watchObservedRunningTime="2026-09-08 20:48:34.728202491 +0000 UTC m=+1388.275452009"
Sep 08 20:48:34.822510 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:34.822477    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:34.916867 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:34.916836    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:34.987463 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:34.987369    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:35.397908 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:35.397875    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:35.500418 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:35.500385    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:43.398052 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:43.398004    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:43.398596 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:43.398082    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:43.410941 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:43.410913    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:43.786142 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:43.786048    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:48:57.848635 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:57.848544    2580 generic.go:356] "Generic (PLEG): container finished" podID="07a0bd22-9061-4fcd-9e3f-0e4aa48af090" containerID="8b547531491d818915a1ae298d01140dbf37986bab0aaeeb95d287677ef27f69" exitCode=0
Sep 08 20:48:57.849164 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:57.848624    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6" event={"ID":"07a0bd22-9061-4fcd-9e3f-0e4aa48af090","Type":"ContainerDied","Data":"8b547531491d818915a1ae298d01140dbf37986bab0aaeeb95d287677ef27f69"}
Sep 08 20:48:57.849230 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:57.849200    2580 scope.go:117] "RemoveContainer" containerID="8b547531491d818915a1ae298d01140dbf37986bab0aaeeb95d287677ef27f69"
Sep 08 20:48:57.850405 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:57.850378    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:48:57.850472 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:57.850444    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:48:57.850511 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:57.850477    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 08 20:48:58.638872 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:58.638833    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:58.864516 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:58.864466    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6" event={"ID":"07a0bd22-9061-4fcd-9e3f-0e4aa48af090","Type":"ContainerStarted","Data":"2e56a0002c48a3a32d7708580f0d9042b146cac5868d65ecbcbf005a2e9a1c5a"}
Sep 08 20:48:58.920149 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:58.920057    2580 patch_prober.go:28] interesting pod/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 08 20:48:58.920149 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:58.920105    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6" podUID="07a0bd22-9061-4fcd-9e3f-0e4aa48af090" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 08 20:48:58.924438 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:58.924414    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:48:59.125338 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:59.125304    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:59.261210 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:59.261107    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:59.463062 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:59.463024    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:48:59.920207 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:59.920169    2580 patch_prober.go:28] interesting pod/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 08 20:48:59.920621 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:48:59.920230    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6" podUID="07a0bd22-9061-4fcd-9e3f-0e4aa48af090" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 08 20:49:00.079623 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:00.079565    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:00.919661 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:00.919632    2580 patch_prober.go:28] interesting pod/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 08 20:49:00.919817 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:00.919696    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6" podUID="07a0bd22-9061-4fcd-9e3f-0e4aa48af090" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 08 20:49:01.920100 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:01.920055    2580 patch_prober.go:28] interesting pod/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 08 20:49:01.920566 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:01.920130    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6" podUID="07a0bd22-9061-4fcd-9e3f-0e4aa48af090" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 08 20:49:02.688726 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:02.688693    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:02.822680 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:02.822651    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:02.919364 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:02.919325    2580 patch_prober.go:28] interesting pod/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 08 20:49:02.919518 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:02.919387    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6" podUID="07a0bd22-9061-4fcd-9e3f-0e4aa48af090" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 08 20:49:03.705620 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:03.705579    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:49:03.909809 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:03.909776    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:04.011229 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:04.011128    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:04.018415 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:04.018375    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:49:04.019750 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:04.019725    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:49:04.122098 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:04.122070    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:04.400763 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:04.400729    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:04.471055 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:04.471018    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:06.533218 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:06.533179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:09.041469 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:09.041439    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:24.823106 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:24.823022    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:27.690177 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:27.690142    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:30.920779 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:30.920748    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:32.281166 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:32.281126    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"]
Sep 08 20:49:32.300311 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:32.300273    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:33.384489 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:33.384450    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"]
Sep 08 20:49:33.404179 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:33.404144    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:36.487826 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:36.487790    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:37.654411 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:37.654375    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:41.035841 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:41.035803    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:46.713918 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:46.713882    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:46.786533 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:46.786496    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:46.845800 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:46.845767    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:46.918322 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:46.918281    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:46.986661 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:46.986578    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:47.279296 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.279204    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:47.302141 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.302101    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww" podUID="c5c4f240-9922-4f58-bac1-1026f9598a1d" containerName="main" containerID="cri-o://abbefa351ecc3179aa7f599181c6179b77537692d85fe6627a634f12a303325a" gracePeriod=45
Sep 08 20:49:47.559311 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.559283    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:49:47.620761 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.620725    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-model-cache\") pod \"c5c4f240-9922-4f58-bac1-1026f9598a1d\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") "
Sep 08 20:49:47.620930 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.620795    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-home\") pod \"c5c4f240-9922-4f58-bac1-1026f9598a1d\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") "
Sep 08 20:49:47.620930 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.620819    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66jnm\" (UniqueName: \"kubernetes.io/projected/c5c4f240-9922-4f58-bac1-1026f9598a1d-kube-api-access-66jnm\") pod \"c5c4f240-9922-4f58-bac1-1026f9598a1d\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") "
Sep 08 20:49:47.620930 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.620853    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-dshm\") pod \"c5c4f240-9922-4f58-bac1-1026f9598a1d\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") "
Sep 08 20:49:47.620930 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.620887    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-tmp-dir\") pod \"c5c4f240-9922-4f58-bac1-1026f9598a1d\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") "
Sep 08 20:49:47.620930 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.620911    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5c4f240-9922-4f58-bac1-1026f9598a1d-tls-certs\") pod \"c5c4f240-9922-4f58-bac1-1026f9598a1d\" (UID: \"c5c4f240-9922-4f58-bac1-1026f9598a1d\") "
Sep 08 20:49:47.621207 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.621089    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-model-cache" (OuterVolumeSpecName: "model-cache") pod "c5c4f240-9922-4f58-bac1-1026f9598a1d" (UID: "c5c4f240-9922-4f58-bac1-1026f9598a1d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:49:47.621207 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.621108    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-home" (OuterVolumeSpecName: "home") pod "c5c4f240-9922-4f58-bac1-1026f9598a1d" (UID: "c5c4f240-9922-4f58-bac1-1026f9598a1d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:49:47.621331 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.621234    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c5c4f240-9922-4f58-bac1-1026f9598a1d" (UID: "c5c4f240-9922-4f58-bac1-1026f9598a1d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:49:47.621456 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.621430    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-tmp-dir\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:47.621456 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.621456    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-model-cache\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:47.621648 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.621469    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-home\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:47.623060 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.623026    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5c4f240-9922-4f58-bac1-1026f9598a1d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c5c4f240-9922-4f58-bac1-1026f9598a1d" (UID: "c5c4f240-9922-4f58-bac1-1026f9598a1d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:49:47.623060 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.623032    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5c4f240-9922-4f58-bac1-1026f9598a1d-kube-api-access-66jnm" (OuterVolumeSpecName: "kube-api-access-66jnm") pod "c5c4f240-9922-4f58-bac1-1026f9598a1d" (UID: "c5c4f240-9922-4f58-bac1-1026f9598a1d"). InnerVolumeSpecName "kube-api-access-66jnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:49:47.623184 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.623077    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-dshm" (OuterVolumeSpecName: "dshm") pod "c5c4f240-9922-4f58-bac1-1026f9598a1d" (UID: "c5c4f240-9922-4f58-bac1-1026f9598a1d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:49:47.722730 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.722690    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66jnm\" (UniqueName: \"kubernetes.io/projected/c5c4f240-9922-4f58-bac1-1026f9598a1d-kube-api-access-66jnm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:47.722730 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.722723    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5c4f240-9922-4f58-bac1-1026f9598a1d-dshm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:47.722730 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:47.722738    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5c4f240-9922-4f58-bac1-1026f9598a1d-tls-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:48.130316 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.130284    2580 generic.go:356] "Generic (PLEG): container finished" podID="c5c4f240-9922-4f58-bac1-1026f9598a1d" containerID="abbefa351ecc3179aa7f599181c6179b77537692d85fe6627a634f12a303325a" exitCode=0
Sep 08 20:49:48.130515 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.130349    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"
Sep 08 20:49:48.130515 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.130354    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww" event={"ID":"c5c4f240-9922-4f58-bac1-1026f9598a1d","Type":"ContainerDied","Data":"abbefa351ecc3179aa7f599181c6179b77537692d85fe6627a634f12a303325a"}
Sep 08 20:49:48.130515 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.130400    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww" event={"ID":"c5c4f240-9922-4f58-bac1-1026f9598a1d","Type":"ContainerDied","Data":"7e665999e507dc5e5e057c5667e9da0279346192991752a15e9dd2d4fc057f8b"}
Sep 08 20:49:48.130515 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.130422    2580 scope.go:117] "RemoveContainer" containerID="abbefa351ecc3179aa7f599181c6179b77537692d85fe6627a634f12a303325a"
Sep 08 20:49:48.149136 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.148687    2580 scope.go:117] "RemoveContainer" containerID="abbefa351ecc3179aa7f599181c6179b77537692d85fe6627a634f12a303325a"
Sep 08 20:49:48.151640 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:49:48.149503    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abbefa351ecc3179aa7f599181c6179b77537692d85fe6627a634f12a303325a\": container with ID starting with abbefa351ecc3179aa7f599181c6179b77537692d85fe6627a634f12a303325a not found: ID does not exist" containerID="abbefa351ecc3179aa7f599181c6179b77537692d85fe6627a634f12a303325a"
Sep 08 20:49:48.151640 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.149539    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abbefa351ecc3179aa7f599181c6179b77537692d85fe6627a634f12a303325a"} err="failed to get container status \"abbefa351ecc3179aa7f599181c6179b77537692d85fe6627a634f12a303325a\": rpc error: code = NotFound desc = could not find container \"abbefa351ecc3179aa7f599181c6179b77537692d85fe6627a634f12a303325a\": container with ID starting with abbefa351ecc3179aa7f599181c6179b77537692d85fe6627a634f12a303325a not found: ID does not exist"
Sep 08 20:49:48.159981 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.159933    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"]
Sep 08 20:49:48.167371 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.167345    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/sub-test-model-78dd9f-kserve-7b79df9575-c9tww"]
Sep 08 20:49:48.404763 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.404653    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9" podUID="41d68009-87dd-42bf-9b29-49f6cf4da23c" containerName="main" containerID="cri-o://ce6cb87aae77d9f21d11c6eaaf517de9a50239af6fa7e4e695d3bfd5b9dfa097" gracePeriod=45
Sep 08 20:49:48.659353 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.659287    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:49:48.731906 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.731857    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-model-cache\") pod \"41d68009-87dd-42bf-9b29-49f6cf4da23c\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") "
Sep 08 20:49:48.731906 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.731911    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41d68009-87dd-42bf-9b29-49f6cf4da23c-tls-certs\") pod \"41d68009-87dd-42bf-9b29-49f6cf4da23c\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") "
Sep 08 20:49:48.732397 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.731985    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mflb\" (UniqueName: \"kubernetes.io/projected/41d68009-87dd-42bf-9b29-49f6cf4da23c-kube-api-access-5mflb\") pod \"41d68009-87dd-42bf-9b29-49f6cf4da23c\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") "
Sep 08 20:49:48.732397 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.732047    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-dshm\") pod \"41d68009-87dd-42bf-9b29-49f6cf4da23c\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") "
Sep 08 20:49:48.732397 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.732068    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-tmp-dir\") pod \"41d68009-87dd-42bf-9b29-49f6cf4da23c\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") "
Sep 08 20:49:48.732397 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.732102    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-home\") pod \"41d68009-87dd-42bf-9b29-49f6cf4da23c\" (UID: \"41d68009-87dd-42bf-9b29-49f6cf4da23c\") "
Sep 08 20:49:48.732397 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.732169    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-model-cache" (OuterVolumeSpecName: "model-cache") pod "41d68009-87dd-42bf-9b29-49f6cf4da23c" (UID: "41d68009-87dd-42bf-9b29-49f6cf4da23c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:49:48.732397 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.732372    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-home" (OuterVolumeSpecName: "home") pod "41d68009-87dd-42bf-9b29-49f6cf4da23c" (UID: "41d68009-87dd-42bf-9b29-49f6cf4da23c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:49:48.732618 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.732399    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "41d68009-87dd-42bf-9b29-49f6cf4da23c" (UID: "41d68009-87dd-42bf-9b29-49f6cf4da23c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:49:48.732618 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.732523    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-model-cache\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:48.732618 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.732566    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-tmp-dir\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:48.732618 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.732580    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-home\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:48.734220 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.734198    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-dshm" (OuterVolumeSpecName: "dshm") pod "41d68009-87dd-42bf-9b29-49f6cf4da23c" (UID: "41d68009-87dd-42bf-9b29-49f6cf4da23c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:49:48.734308 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.734199    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41d68009-87dd-42bf-9b29-49f6cf4da23c-kube-api-access-5mflb" (OuterVolumeSpecName: "kube-api-access-5mflb") pod "41d68009-87dd-42bf-9b29-49f6cf4da23c" (UID: "41d68009-87dd-42bf-9b29-49f6cf4da23c"). InnerVolumeSpecName "kube-api-access-5mflb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:49:48.734308 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.734223    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41d68009-87dd-42bf-9b29-49f6cf4da23c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "41d68009-87dd-42bf-9b29-49f6cf4da23c" (UID: "41d68009-87dd-42bf-9b29-49f6cf4da23c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:49:48.833798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.833758    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mflb\" (UniqueName: \"kubernetes.io/projected/41d68009-87dd-42bf-9b29-49f6cf4da23c-kube-api-access-5mflb\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:48.833798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.833792    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41d68009-87dd-42bf-9b29-49f6cf4da23c-dshm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:48.833798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.833802    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41d68009-87dd-42bf-9b29-49f6cf4da23c-tls-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:48.924649 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:48.924555    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5c4f240-9922-4f58-bac1-1026f9598a1d" path="/var/lib/kubelet/pods/c5c4f240-9922-4f58-bac1-1026f9598a1d/volumes"
Sep 08 20:49:49.136931 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:49.136891    2580 generic.go:356] "Generic (PLEG): container finished" podID="41d68009-87dd-42bf-9b29-49f6cf4da23c" containerID="ce6cb87aae77d9f21d11c6eaaf517de9a50239af6fa7e4e695d3bfd5b9dfa097" exitCode=0
Sep 08 20:49:49.136931 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:49.136933    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"
Sep 08 20:49:49.137186 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:49.136999    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9" event={"ID":"41d68009-87dd-42bf-9b29-49f6cf4da23c","Type":"ContainerDied","Data":"ce6cb87aae77d9f21d11c6eaaf517de9a50239af6fa7e4e695d3bfd5b9dfa097"}
Sep 08 20:49:49.137186 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:49.137046    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9" event={"ID":"41d68009-87dd-42bf-9b29-49f6cf4da23c","Type":"ContainerDied","Data":"c244fa1b7f538061cda43ace76bd190b703655b4078d87b1b727f5756e890ad2"}
Sep 08 20:49:49.137186 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:49.137069    2580 scope.go:117] "RemoveContainer" containerID="ce6cb87aae77d9f21d11c6eaaf517de9a50239af6fa7e4e695d3bfd5b9dfa097"
Sep 08 20:49:49.145901 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:49.145883    2580 scope.go:117] "RemoveContainer" containerID="ce6cb87aae77d9f21d11c6eaaf517de9a50239af6fa7e4e695d3bfd5b9dfa097"
Sep 08 20:49:49.146205 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:49:49.146180    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce6cb87aae77d9f21d11c6eaaf517de9a50239af6fa7e4e695d3bfd5b9dfa097\": container with ID starting with ce6cb87aae77d9f21d11c6eaaf517de9a50239af6fa7e4e695d3bfd5b9dfa097 not found: ID does not exist" containerID="ce6cb87aae77d9f21d11c6eaaf517de9a50239af6fa7e4e695d3bfd5b9dfa097"
Sep 08 20:49:49.146306 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:49.146210    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce6cb87aae77d9f21d11c6eaaf517de9a50239af6fa7e4e695d3bfd5b9dfa097"} err="failed to get container status \"ce6cb87aae77d9f21d11c6eaaf517de9a50239af6fa7e4e695d3bfd5b9dfa097\": rpc error: code = NotFound desc = could not find container \"ce6cb87aae77d9f21d11c6eaaf517de9a50239af6fa7e4e695d3bfd5b9dfa097\": container with ID starting with ce6cb87aae77d9f21d11c6eaaf517de9a50239af6fa7e4e695d3bfd5b9dfa097 not found: ID does not exist"
Sep 08 20:49:49.153874 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:49.153850    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"]
Sep 08 20:49:49.164386 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:49.164356    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/sub-test-model-c9590b-kserve-c5bfdfb79-mzkr9"]
Sep 08 20:49:50.924870 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:50.924833    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41d68009-87dd-42bf-9b29-49f6cf4da23c" path="/var/lib/kubelet/pods/41d68009-87dd-42bf-9b29-49f6cf4da23c/volumes"
Sep 08 20:49:51.077448 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.077412    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"]
Sep 08 20:49:51.077825 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.077809    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5c4f240-9922-4f58-bac1-1026f9598a1d" containerName="main"
Sep 08 20:49:51.077869 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.077828    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5c4f240-9922-4f58-bac1-1026f9598a1d" containerName="main"
Sep 08 20:49:51.077910 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.077872    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41d68009-87dd-42bf-9b29-49f6cf4da23c" containerName="main"
Sep 08 20:49:51.077910 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.077880    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="41d68009-87dd-42bf-9b29-49f6cf4da23c" containerName="main"
Sep 08 20:49:51.078006 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.077977    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5c4f240-9922-4f58-bac1-1026f9598a1d" containerName="main"
Sep 08 20:49:51.078006 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.077989    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="41d68009-87dd-42bf-9b29-49f6cf4da23c" containerName="main"
Sep 08 20:49:51.081062 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.081040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.083288 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.083260    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-78dd9f\"/\"openshift-service-ca.crt\""
Sep 08 20:49:51.083428 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.083300    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-78dd9f\"/\"kube-root-ca.crt\""
Sep 08 20:49:51.083428 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.083317    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-78dd9f\"/\"default-dockercfg-r8s7w\""
Sep 08 20:49:51.083939 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.083916    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-78dd9f\"/\"rate-test-model-78dd9f-kserve-self-signed-certs\""
Sep 08 20:49:51.093216 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.093192    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"]
Sep 08 20:49:51.104905 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.104866    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:51.155393 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.155363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-model-cache\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.155597 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.155407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-home\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.155597 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.155496    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-tmp-dir\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.155597 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.155529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7gpf\" (UniqueName: \"kubernetes.io/projected/8b5708e0-de84-4800-a8d7-dffa2008d61e-kube-api-access-x7gpf\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.155597 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.155556    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-dshm\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.155597 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.155577    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b5708e0-de84-4800-a8d7-dffa2008d61e-tls-certs\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.256813 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.256707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-tmp-dir\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.256813 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.256771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7gpf\" (UniqueName: \"kubernetes.io/projected/8b5708e0-de84-4800-a8d7-dffa2008d61e-kube-api-access-x7gpf\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.257083 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.256814    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-dshm\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.257083 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.256842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b5708e0-de84-4800-a8d7-dffa2008d61e-tls-certs\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.257083 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.256894    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-model-cache\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.257083 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.256937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-home\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.257469 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.257445    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-home\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.257872 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.257850    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-model-cache\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.258034 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.257899    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-tmp-dir\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.259860 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.259806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-dshm\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.260540 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.260515    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b5708e0-de84-4800-a8d7-dffa2008d61e-tls-certs\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.265304 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.265269    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7gpf\" (UniqueName: \"kubernetes.io/projected/8b5708e0-de84-4800-a8d7-dffa2008d61e-kube-api-access-x7gpf\") pod \"rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") " pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.390413 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.390378    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:49:51.661677 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:51.661646    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"]
Sep 08 20:49:51.663385 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:49:51.663359    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b5708e0_de84_4800_a8d7_dffa2008d61e.slice/crio-87d7fcb95ebcb003b9619371b3f0e88880dd399c4b444428a58d1eba5b5f4d64 WatchSource:0}: Error finding container 87d7fcb95ebcb003b9619371b3f0e88880dd399c4b444428a58d1eba5b5f4d64: Status 404 returned error can't find the container with id 87d7fcb95ebcb003b9619371b3f0e88880dd399c4b444428a58d1eba5b5f4d64
Sep 08 20:49:52.157789 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:52.157757    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2" event={"ID":"8b5708e0-de84-4800-a8d7-dffa2008d61e","Type":"ContainerStarted","Data":"a51fe1d28546faf0fe9a07bf1feb8fe79896b42a9a3919053dad2cd4b3cc0d4c"}
Sep 08 20:49:52.157789 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:52.157795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2" event={"ID":"8b5708e0-de84-4800-a8d7-dffa2008d61e","Type":"ContainerStarted","Data":"87d7fcb95ebcb003b9619371b3f0e88880dd399c4b444428a58d1eba5b5f4d64"}
Sep 08 20:49:52.176782 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:52.176732    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2" podStartSLOduration=1.17671698 podStartE2EDuration="1.17671698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:49:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:49:52.175112885 +0000 UTC m=+1465.722362380" watchObservedRunningTime="2026-09-08 20:49:52.17671698 +0000 UTC m=+1465.723966462"
Sep 08 20:49:53.200195 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:53.200157    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:53.272518 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:53.272483    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:53.446933 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:53.446895    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"]
Sep 08 20:49:53.447190 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:53.447164    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6" podUID="07a0bd22-9061-4fcd-9e3f-0e4aa48af090" containerName="istio-proxy" containerID="cri-o://2e56a0002c48a3a32d7708580f0d9042b146cac5868d65ecbcbf005a2e9a1c5a" gracePeriod=30
Sep 08 20:49:53.467684 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:53.467592    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:53.939481 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:53.939443    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:49:58.705168 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.705143    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:49:58.837844 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.837759    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-workload-certs\") pod \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") "
Sep 08 20:49:58.837844 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.837800    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-data\") pod \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") "
Sep 08 20:49:58.838120 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.837875    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-token\") pod \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") "
Sep 08 20:49:58.838120 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.837922    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-workload-socket\") pod \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") "
Sep 08 20:49:58.838120 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.838000    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istiod-ca-cert\") pod \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") "
Sep 08 20:49:58.838120 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.838049    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-envoy\") pod \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") "
Sep 08 20:49:58.838120 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.838118    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-podinfo\") pod \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") "
Sep 08 20:49:58.838382 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.838124    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "07a0bd22-9061-4fcd-9e3f-0e4aa48af090" (UID: "07a0bd22-9061-4fcd-9e3f-0e4aa48af090"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:49:58.838382 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.838151    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-credential-socket\") pod \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") "
Sep 08 20:49:58.838382 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.838175    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-data" (OuterVolumeSpecName: "istio-data") pod "07a0bd22-9061-4fcd-9e3f-0e4aa48af090" (UID: "07a0bd22-9061-4fcd-9e3f-0e4aa48af090"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:49:58.838382 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.838192    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddq2r\" (UniqueName: \"kubernetes.io/projected/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-kube-api-access-ddq2r\") pod \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\" (UID: \"07a0bd22-9061-4fcd-9e3f-0e4aa48af090\") "
Sep 08 20:49:58.838597 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.838395    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "07a0bd22-9061-4fcd-9e3f-0e4aa48af090" (UID: "07a0bd22-9061-4fcd-9e3f-0e4aa48af090"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:49:58.838654 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.838606    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "07a0bd22-9061-4fcd-9e3f-0e4aa48af090" (UID: "07a0bd22-9061-4fcd-9e3f-0e4aa48af090"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:49:58.838654 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.838616    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-credential-socket\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:58.838765 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.838659    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-workload-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:58.838765 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.838676    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-data\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:58.838765 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.838707    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "07a0bd22-9061-4fcd-9e3f-0e4aa48af090" (UID: "07a0bd22-9061-4fcd-9e3f-0e4aa48af090"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:49:58.840376 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.840348    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "07a0bd22-9061-4fcd-9e3f-0e4aa48af090" (UID: "07a0bd22-9061-4fcd-9e3f-0e4aa48af090"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:49:58.840473 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.840352    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-token" (OuterVolumeSpecName: "istio-token") pod "07a0bd22-9061-4fcd-9e3f-0e4aa48af090" (UID: "07a0bd22-9061-4fcd-9e3f-0e4aa48af090"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:49:58.840660 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.840638    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-kube-api-access-ddq2r" (OuterVolumeSpecName: "kube-api-access-ddq2r") pod "07a0bd22-9061-4fcd-9e3f-0e4aa48af090" (UID: "07a0bd22-9061-4fcd-9e3f-0e4aa48af090"). InnerVolumeSpecName "kube-api-access-ddq2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:49:58.840731 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.840641    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "07a0bd22-9061-4fcd-9e3f-0e4aa48af090" (UID: "07a0bd22-9061-4fcd-9e3f-0e4aa48af090"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:49:58.939752 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.939727    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-podinfo\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:58.939856 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.939763    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddq2r\" (UniqueName: \"kubernetes.io/projected/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-kube-api-access-ddq2r\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:58.939856 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.939773    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-token\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:58.939856 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.939782    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-workload-socket\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:58.939856 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.939791    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istiod-ca-cert\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:58.939856 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:58.939799    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07a0bd22-9061-4fcd-9e3f-0e4aa48af090-istio-envoy\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:49:59.197031 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:59.196996    2580 generic.go:356] "Generic (PLEG): container finished" podID="07a0bd22-9061-4fcd-9e3f-0e4aa48af090" containerID="2e56a0002c48a3a32d7708580f0d9042b146cac5868d65ecbcbf005a2e9a1c5a" exitCode=0
Sep 08 20:49:59.197209 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:59.197061    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"
Sep 08 20:49:59.197209 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:59.197089    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6" event={"ID":"07a0bd22-9061-4fcd-9e3f-0e4aa48af090","Type":"ContainerDied","Data":"2e56a0002c48a3a32d7708580f0d9042b146cac5868d65ecbcbf005a2e9a1c5a"}
Sep 08 20:49:59.197209 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:59.197137    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6" event={"ID":"07a0bd22-9061-4fcd-9e3f-0e4aa48af090","Type":"ContainerDied","Data":"5f2d68858c8103eefb52d944917a45eb83df81eb9c7451d492b1c025ed75bd0e"}
Sep 08 20:49:59.197209 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:59.197160    2580 scope.go:117] "RemoveContainer" containerID="2e56a0002c48a3a32d7708580f0d9042b146cac5868d65ecbcbf005a2e9a1c5a"
Sep 08 20:49:59.206268 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:59.206245    2580 scope.go:117] "RemoveContainer" containerID="8b547531491d818915a1ae298d01140dbf37986bab0aaeeb95d287677ef27f69"
Sep 08 20:49:59.214206 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:59.214188    2580 scope.go:117] "RemoveContainer" containerID="2e56a0002c48a3a32d7708580f0d9042b146cac5868d65ecbcbf005a2e9a1c5a"
Sep 08 20:49:59.214464 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:49:59.214447    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e56a0002c48a3a32d7708580f0d9042b146cac5868d65ecbcbf005a2e9a1c5a\": container with ID starting with 2e56a0002c48a3a32d7708580f0d9042b146cac5868d65ecbcbf005a2e9a1c5a not found: ID does not exist" containerID="2e56a0002c48a3a32d7708580f0d9042b146cac5868d65ecbcbf005a2e9a1c5a"
Sep 08 20:49:59.214506 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:59.214473    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e56a0002c48a3a32d7708580f0d9042b146cac5868d65ecbcbf005a2e9a1c5a"} err="failed to get container status \"2e56a0002c48a3a32d7708580f0d9042b146cac5868d65ecbcbf005a2e9a1c5a\": rpc error: code = NotFound desc = could not find container \"2e56a0002c48a3a32d7708580f0d9042b146cac5868d65ecbcbf005a2e9a1c5a\": container with ID starting with 2e56a0002c48a3a32d7708580f0d9042b146cac5868d65ecbcbf005a2e9a1c5a not found: ID does not exist"
Sep 08 20:49:59.214506 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:59.214489    2580 scope.go:117] "RemoveContainer" containerID="8b547531491d818915a1ae298d01140dbf37986bab0aaeeb95d287677ef27f69"
Sep 08 20:49:59.214748 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:49:59.214733    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b547531491d818915a1ae298d01140dbf37986bab0aaeeb95d287677ef27f69\": container with ID starting with 8b547531491d818915a1ae298d01140dbf37986bab0aaeeb95d287677ef27f69 not found: ID does not exist" containerID="8b547531491d818915a1ae298d01140dbf37986bab0aaeeb95d287677ef27f69"
Sep 08 20:49:59.214802 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:59.214750    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b547531491d818915a1ae298d01140dbf37986bab0aaeeb95d287677ef27f69"} err="failed to get container status \"8b547531491d818915a1ae298d01140dbf37986bab0aaeeb95d287677ef27f69\": rpc error: code = NotFound desc = could not find container \"8b547531491d818915a1ae298d01140dbf37986bab0aaeeb95d287677ef27f69\": container with ID starting with 8b547531491d818915a1ae298d01140dbf37986bab0aaeeb95d287677ef27f69 not found: ID does not exist"
Sep 08 20:49:59.218400 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:59.218378    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"]
Sep 08 20:49:59.222501 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:49:59.222478    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ce2cc8-openshift-default-5cb75b6dfb-jmkp6"]
Sep 08 20:50:00.924402 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:00.924365    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07a0bd22-9061-4fcd-9e3f-0e4aa48af090" path="/var/lib/kubelet/pods/07a0bd22-9061-4fcd-9e3f-0e4aa48af090/volumes"
Sep 08 20:50:01.391449 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:01.391416    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:50:01.391607 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:01.391469    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:50:01.404527 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:01.404493    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:50:02.225723 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:02.225693    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:50:09.327722 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:09.327681    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:09.393541 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:09.393497    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:09.455696 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:09.455659    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:09.518930 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:09.518895    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:09.581040 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:09.580931    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:09.897075 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:09.897047    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:14.979866 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:14.979827    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:15.737112 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:15.737077    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:15.852537 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:15.852499    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:16.025025 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:16.024921    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:16.512098 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:16.512052    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:22.388269 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.388232    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"]
Sep 08 20:50:22.388659 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.388645    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07a0bd22-9061-4fcd-9e3f-0e4aa48af090" containerName="istio-proxy"
Sep 08 20:50:22.388703 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.388660    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="07a0bd22-9061-4fcd-9e3f-0e4aa48af090" containerName="istio-proxy"
Sep 08 20:50:22.388751 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.388741    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="07a0bd22-9061-4fcd-9e3f-0e4aa48af090" containerName="istio-proxy"
Sep 08 20:50:22.388787 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.388752    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="07a0bd22-9061-4fcd-9e3f-0e4aa48af090" containerName="istio-proxy"
Sep 08 20:50:22.393258 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.393231    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.396086 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.396058    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-c9590b\"/\"openshift-service-ca.crt\""
Sep 08 20:50:22.396218 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.396058    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-c9590b\"/\"rate-test-model-c9590b-kserve-self-signed-certs\""
Sep 08 20:50:22.396218 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.396061    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-c9590b\"/\"kube-root-ca.crt\""
Sep 08 20:50:22.396218 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.396142    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-c9590b\"/\"default-dockercfg-29kxk\""
Sep 08 20:50:22.401019 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.400865    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"]
Sep 08 20:50:22.417274 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.417244    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:22.468341 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.468309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-tmp-dir\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.468341 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.468347    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-model-cache\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.468601 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.468485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r785j\" (UniqueName: \"kubernetes.io/projected/abce160c-cd67-4c0d-8a78-19268ae612f5-kube-api-access-r785j\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.468601 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.468524    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-dshm\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.468601 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.468561    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-home\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.468601 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.468577    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abce160c-cd67-4c0d-8a78-19268ae612f5-tls-certs\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.569587 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.569545    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-dshm\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.569587 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.569588    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-home\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.569822 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.569607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abce160c-cd67-4c0d-8a78-19268ae612f5-tls-certs\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.569822 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.569689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-tmp-dir\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.569822 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.569723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-model-cache\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.570011 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.569829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r785j\" (UniqueName: \"kubernetes.io/projected/abce160c-cd67-4c0d-8a78-19268ae612f5-kube-api-access-r785j\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.570118 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.570094    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-home\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.570179 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.570145    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-tmp-dir\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.570225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.570196    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-model-cache\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.571833 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.571816    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-dshm\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.572198 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.572181    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abce160c-cd67-4c0d-8a78-19268ae612f5-tls-certs\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.577470 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.577448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r785j\" (UniqueName: \"kubernetes.io/projected/abce160c-cd67-4c0d-8a78-19268ae612f5-kube-api-access-r785j\") pod \"rate-test-model-c9590b-kserve-65b689874c-fbdlz\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") " pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.704087 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.703994    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:22.986468 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:22.986440    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"]
Sep 08 20:50:22.988650 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:50:22.988623    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabce160c_cd67_4c0d_8a78_19268ae612f5.slice/crio-cedf1ffdb2f9bf65e5190fc5997b1a607607434b99480cc97bd75843054ffed6 WatchSource:0}: Error finding container cedf1ffdb2f9bf65e5190fc5997b1a607607434b99480cc97bd75843054ffed6: Status 404 returned error can't find the container with id cedf1ffdb2f9bf65e5190fc5997b1a607607434b99480cc97bd75843054ffed6
Sep 08 20:50:23.325161 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:23.325051    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:23.325161 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:23.325076    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz" event={"ID":"abce160c-cd67-4c0d-8a78-19268ae612f5","Type":"ContainerStarted","Data":"212b8383a2a5206b7d12d67ef4121bb7b4685b5e69f6c19dd8dc2afd10ebcb83"}
Sep 08 20:50:23.325161 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:23.325116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz" event={"ID":"abce160c-cd67-4c0d-8a78-19268ae612f5","Type":"ContainerStarted","Data":"cedf1ffdb2f9bf65e5190fc5997b1a607607434b99480cc97bd75843054ffed6"}
Sep 08 20:50:23.345990 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:23.345904    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz" podStartSLOduration=1.345887781 podStartE2EDuration="1.345887781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:50:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:50:23.343681139 +0000 UTC m=+1496.890930622" watchObservedRunningTime="2026-09-08 20:50:23.345887781 +0000 UTC m=+1496.893137303"
Sep 08 20:50:26.524772 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:26.524729    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:27.077840 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:27.077809    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:27.154234 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:27.154189    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:27.219907 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:27.219877    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:27.289548 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:27.289511    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:27.353455 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:27.353373    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:27.670891 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:27.670861    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:32.704460 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:32.704415    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:32.704994 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:32.704479    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:32.717069 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:32.717040    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:33.392884 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:33.392849    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:50:33.642981 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:33.642874    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:33.743138 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:33.743106    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:33.917618 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:33.917536    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:34.434011 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:34.433945    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:44.851775 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:44.851731    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-74f9a1-29815005-qc86w"]
Sep 08 20:50:44.854137 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:44.854107    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-74f9a1-29815005-qc86w"]
Sep 08 20:50:44.925340 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:44.925308    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea2ab14b-44f2-4d43-9113-345584fd63f3" path="/var/lib/kubelet/pods/ea2ab14b-44f2-4d43-9113-345584fd63f3/volumes"
Sep 08 20:50:45.662104 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:45.662066    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:45.727865 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:45.727833    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:51.151632 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:51.151587    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:51.258458 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:51.258415    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:51.411072 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:51.410982    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"]
Sep 08 20:50:51.411229 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:51.411193    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7" podUID="e28c4038-41cf-46fd-91e3-410d097ca036" containerName="istio-proxy" containerID="cri-o://ab0ec76415b5b45540071b930eff353509b299ff722e1a827e34a47b1cd9aaad" gracePeriod=30
Sep 08 20:50:51.440703 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:51.440670    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:50:53.094849 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:53.094811    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 20:50:53.095258 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:53.094882    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7" podUID="e28c4038-41cf-46fd-91e3-410d097ca036" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 20:50:56.670533 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.670506    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:50:56.720484 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.720399    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-workload-socket\") pod \"e28c4038-41cf-46fd-91e3-410d097ca036\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") "
Sep 08 20:50:56.720484 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.720449    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e28c4038-41cf-46fd-91e3-410d097ca036-istio-podinfo\") pod \"e28c4038-41cf-46fd-91e3-410d097ca036\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") "
Sep 08 20:50:56.720484 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.720468    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e28c4038-41cf-46fd-91e3-410d097ca036-istio-token\") pod \"e28c4038-41cf-46fd-91e3-410d097ca036\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") "
Sep 08 20:50:56.720735 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.720498    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-credential-socket\") pod \"e28c4038-41cf-46fd-91e3-410d097ca036\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") "
Sep 08 20:50:56.720735 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.720544    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-istio-data\") pod \"e28c4038-41cf-46fd-91e3-410d097ca036\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") "
Sep 08 20:50:56.720735 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.720566    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zzc5\" (UniqueName: \"kubernetes.io/projected/e28c4038-41cf-46fd-91e3-410d097ca036-kube-api-access-8zzc5\") pod \"e28c4038-41cf-46fd-91e3-410d097ca036\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") "
Sep 08 20:50:56.720735 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.720602    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e28c4038-41cf-46fd-91e3-410d097ca036-istiod-ca-cert\") pod \"e28c4038-41cf-46fd-91e3-410d097ca036\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") "
Sep 08 20:50:56.720735 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.720630    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-istio-envoy\") pod \"e28c4038-41cf-46fd-91e3-410d097ca036\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") "
Sep 08 20:50:56.720735 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.720689    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-workload-certs\") pod \"e28c4038-41cf-46fd-91e3-410d097ca036\" (UID: \"e28c4038-41cf-46fd-91e3-410d097ca036\") "
Sep 08 20:50:56.721091 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.720748    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e28c4038-41cf-46fd-91e3-410d097ca036" (UID: "e28c4038-41cf-46fd-91e3-410d097ca036"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:50:56.721091 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.720841    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e28c4038-41cf-46fd-91e3-410d097ca036" (UID: "e28c4038-41cf-46fd-91e3-410d097ca036"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:50:56.721091 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.720865    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-istio-data" (OuterVolumeSpecName: "istio-data") pod "e28c4038-41cf-46fd-91e3-410d097ca036" (UID: "e28c4038-41cf-46fd-91e3-410d097ca036"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:50:56.721091 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.721052    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e28c4038-41cf-46fd-91e3-410d097ca036" (UID: "e28c4038-41cf-46fd-91e3-410d097ca036"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:50:56.721311 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.721094    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e28c4038-41cf-46fd-91e3-410d097ca036-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e28c4038-41cf-46fd-91e3-410d097ca036" (UID: "e28c4038-41cf-46fd-91e3-410d097ca036"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:50:56.721311 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.721164    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-istio-data\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:50:56.721311 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.721176    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e28c4038-41cf-46fd-91e3-410d097ca036-istiod-ca-cert\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:50:56.721311 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.721184    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-workload-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:50:56.721311 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.721192    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-workload-socket\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:50:56.721311 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.721200    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-credential-socket\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:50:56.722758 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.722726    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e28c4038-41cf-46fd-91e3-410d097ca036-istio-token" (OuterVolumeSpecName: "istio-token") pod "e28c4038-41cf-46fd-91e3-410d097ca036" (UID: "e28c4038-41cf-46fd-91e3-410d097ca036"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:50:56.722877 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.722830    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e28c4038-41cf-46fd-91e3-410d097ca036-kube-api-access-8zzc5" (OuterVolumeSpecName: "kube-api-access-8zzc5") pod "e28c4038-41cf-46fd-91e3-410d097ca036" (UID: "e28c4038-41cf-46fd-91e3-410d097ca036"). InnerVolumeSpecName "kube-api-access-8zzc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:50:56.723165 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.723142    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e28c4038-41cf-46fd-91e3-410d097ca036" (UID: "e28c4038-41cf-46fd-91e3-410d097ca036"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:50:56.723260 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.723190    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e28c4038-41cf-46fd-91e3-410d097ca036-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e28c4038-41cf-46fd-91e3-410d097ca036" (UID: "e28c4038-41cf-46fd-91e3-410d097ca036"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:50:56.821846 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.821804    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zzc5\" (UniqueName: \"kubernetes.io/projected/e28c4038-41cf-46fd-91e3-410d097ca036-kube-api-access-8zzc5\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:50:56.821846 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.821839    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e28c4038-41cf-46fd-91e3-410d097ca036-istio-envoy\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:50:56.821846 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.821848    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e28c4038-41cf-46fd-91e3-410d097ca036-istio-podinfo\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:50:56.821846 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:56.821857    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e28c4038-41cf-46fd-91e3-410d097ca036-istio-token\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:50:57.510421 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:57.510384    2580 generic.go:356] "Generic (PLEG): container finished" podID="e28c4038-41cf-46fd-91e3-410d097ca036" containerID="ab0ec76415b5b45540071b930eff353509b299ff722e1a827e34a47b1cd9aaad" exitCode=0
Sep 08 20:50:57.510627 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:57.510440    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"
Sep 08 20:50:57.510627 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:57.510445    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7" event={"ID":"e28c4038-41cf-46fd-91e3-410d097ca036","Type":"ContainerDied","Data":"ab0ec76415b5b45540071b930eff353509b299ff722e1a827e34a47b1cd9aaad"}
Sep 08 20:50:57.510627 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:57.510559    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7" event={"ID":"e28c4038-41cf-46fd-91e3-410d097ca036","Type":"ContainerDied","Data":"f4db85e4f0354503fb5f3d063076abef7f9ab8c4d5cef793420c6f4487893f11"}
Sep 08 20:50:57.510627 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:57.510581    2580 scope.go:117] "RemoveContainer" containerID="ab0ec76415b5b45540071b930eff353509b299ff722e1a827e34a47b1cd9aaad"
Sep 08 20:50:57.519532 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:57.519514    2580 scope.go:117] "RemoveContainer" containerID="fc2c8c186b06c17034810a702a239bb0a50ced52c2c222b47980546e1203856f"
Sep 08 20:50:57.528130 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:57.528106    2580 scope.go:117] "RemoveContainer" containerID="ab0ec76415b5b45540071b930eff353509b299ff722e1a827e34a47b1cd9aaad"
Sep 08 20:50:57.528433 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:50:57.528414    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab0ec76415b5b45540071b930eff353509b299ff722e1a827e34a47b1cd9aaad\": container with ID starting with ab0ec76415b5b45540071b930eff353509b299ff722e1a827e34a47b1cd9aaad not found: ID does not exist" containerID="ab0ec76415b5b45540071b930eff353509b299ff722e1a827e34a47b1cd9aaad"
Sep 08 20:50:57.528510 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:57.528442    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab0ec76415b5b45540071b930eff353509b299ff722e1a827e34a47b1cd9aaad"} err="failed to get container status \"ab0ec76415b5b45540071b930eff353509b299ff722e1a827e34a47b1cd9aaad\": rpc error: code = NotFound desc = could not find container \"ab0ec76415b5b45540071b930eff353509b299ff722e1a827e34a47b1cd9aaad\": container with ID starting with ab0ec76415b5b45540071b930eff353509b299ff722e1a827e34a47b1cd9aaad not found: ID does not exist"
Sep 08 20:50:57.528510 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:57.528460    2580 scope.go:117] "RemoveContainer" containerID="fc2c8c186b06c17034810a702a239bb0a50ced52c2c222b47980546e1203856f"
Sep 08 20:50:57.528705 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:50:57.528684    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc2c8c186b06c17034810a702a239bb0a50ced52c2c222b47980546e1203856f\": container with ID starting with fc2c8c186b06c17034810a702a239bb0a50ced52c2c222b47980546e1203856f not found: ID does not exist" containerID="fc2c8c186b06c17034810a702a239bb0a50ced52c2c222b47980546e1203856f"
Sep 08 20:50:57.528763 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:57.528710    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc2c8c186b06c17034810a702a239bb0a50ced52c2c222b47980546e1203856f"} err="failed to get container status \"fc2c8c186b06c17034810a702a239bb0a50ced52c2c222b47980546e1203856f\": rpc error: code = NotFound desc = could not find container \"fc2c8c186b06c17034810a702a239bb0a50ced52c2c222b47980546e1203856f\": container with ID starting with fc2c8c186b06c17034810a702a239bb0a50ced52c2c222b47980546e1203856f not found: ID does not exist"
Sep 08 20:50:57.529029 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:57.529008    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"]
Sep 08 20:50:57.532970 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:57.532939    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-74f9a1-openshift-default-856d98b9dd-6c7k7"]
Sep 08 20:50:58.924816 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:50:58.924757    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e28c4038-41cf-46fd-91e3-410d097ca036" path="/var/lib/kubelet/pods/e28c4038-41cf-46fd-91e3-410d097ca036/volumes"
Sep 08 20:51:02.513421 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:02.513373    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"]
Sep 08 20:51:02.534617 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:02.534588    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:03.474385 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:03.474342    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"]
Sep 08 20:51:03.496677 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:03.496639    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:03.940134 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:03.940097    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:04.350624 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:04.350536    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:04.619212 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:04.619111    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:07.654771 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:07.654734    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:07.731363 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:07.731322    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:08.803264 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:08.803175    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:09.428947 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:09.428899    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:09.526486 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:09.526444    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:17.533559 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.533518    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2" podUID="8b5708e0-de84-4800-a8d7-dffa2008d61e" containerName="main" containerID="cri-o://a51fe1d28546faf0fe9a07bf1feb8fe79896b42a9a3919053dad2cd4b3cc0d4c" gracePeriod=45
Sep 08 20:51:17.787505 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.787430    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:51:17.842545 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.842511    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-model-cache\") pod \"8b5708e0-de84-4800-a8d7-dffa2008d61e\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") "
Sep 08 20:51:17.842722 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.842560    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-home\") pod \"8b5708e0-de84-4800-a8d7-dffa2008d61e\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") "
Sep 08 20:51:17.842722 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.842593    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b5708e0-de84-4800-a8d7-dffa2008d61e-tls-certs\") pod \"8b5708e0-de84-4800-a8d7-dffa2008d61e\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") "
Sep 08 20:51:17.842722 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.842634    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-tmp-dir\") pod \"8b5708e0-de84-4800-a8d7-dffa2008d61e\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") "
Sep 08 20:51:17.842722 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.842667    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-dshm\") pod \"8b5708e0-de84-4800-a8d7-dffa2008d61e\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") "
Sep 08 20:51:17.842987 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.842736    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7gpf\" (UniqueName: \"kubernetes.io/projected/8b5708e0-de84-4800-a8d7-dffa2008d61e-kube-api-access-x7gpf\") pod \"8b5708e0-de84-4800-a8d7-dffa2008d61e\" (UID: \"8b5708e0-de84-4800-a8d7-dffa2008d61e\") "
Sep 08 20:51:17.842987 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.842845    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-model-cache" (OuterVolumeSpecName: "model-cache") pod "8b5708e0-de84-4800-a8d7-dffa2008d61e" (UID: "8b5708e0-de84-4800-a8d7-dffa2008d61e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:51:17.842987 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.842911    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-home" (OuterVolumeSpecName: "home") pod "8b5708e0-de84-4800-a8d7-dffa2008d61e" (UID: "8b5708e0-de84-4800-a8d7-dffa2008d61e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:51:17.842987 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.842930    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8b5708e0-de84-4800-a8d7-dffa2008d61e" (UID: "8b5708e0-de84-4800-a8d7-dffa2008d61e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:51:17.843265 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.843246    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-home\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:51:17.843334 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.843270    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-tmp-dir\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:51:17.843334 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.843285    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-model-cache\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:51:17.844871 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.844848    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b5708e0-de84-4800-a8d7-dffa2008d61e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8b5708e0-de84-4800-a8d7-dffa2008d61e" (UID: "8b5708e0-de84-4800-a8d7-dffa2008d61e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:51:17.844871 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.844846    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-dshm" (OuterVolumeSpecName: "dshm") pod "8b5708e0-de84-4800-a8d7-dffa2008d61e" (UID: "8b5708e0-de84-4800-a8d7-dffa2008d61e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:51:17.845031 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.844868    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b5708e0-de84-4800-a8d7-dffa2008d61e-kube-api-access-x7gpf" (OuterVolumeSpecName: "kube-api-access-x7gpf") pod "8b5708e0-de84-4800-a8d7-dffa2008d61e" (UID: "8b5708e0-de84-4800-a8d7-dffa2008d61e"). InnerVolumeSpecName "kube-api-access-x7gpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:51:17.943972 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.943930    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7gpf\" (UniqueName: \"kubernetes.io/projected/8b5708e0-de84-4800-a8d7-dffa2008d61e-kube-api-access-x7gpf\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:51:17.944145 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.943994    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b5708e0-de84-4800-a8d7-dffa2008d61e-tls-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:51:17.944145 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:17.944008    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b5708e0-de84-4800-a8d7-dffa2008d61e-dshm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:51:18.495350 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.495289    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz" podUID="abce160c-cd67-4c0d-8a78-19268ae612f5" containerName="main" containerID="cri-o://212b8383a2a5206b7d12d67ef4121bb7b4685b5e69f6c19dd8dc2afd10ebcb83" gracePeriod=45
Sep 08 20:51:18.627713 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.627683    2580 generic.go:356] "Generic (PLEG): container finished" podID="8b5708e0-de84-4800-a8d7-dffa2008d61e" containerID="a51fe1d28546faf0fe9a07bf1feb8fe79896b42a9a3919053dad2cd4b3cc0d4c" exitCode=0
Sep 08 20:51:18.628144 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.627729    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"
Sep 08 20:51:18.628144 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.627762    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2" event={"ID":"8b5708e0-de84-4800-a8d7-dffa2008d61e","Type":"ContainerDied","Data":"a51fe1d28546faf0fe9a07bf1feb8fe79896b42a9a3919053dad2cd4b3cc0d4c"}
Sep 08 20:51:18.628144 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.627811    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2" event={"ID":"8b5708e0-de84-4800-a8d7-dffa2008d61e","Type":"ContainerDied","Data":"87d7fcb95ebcb003b9619371b3f0e88880dd399c4b444428a58d1eba5b5f4d64"}
Sep 08 20:51:18.628144 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.627835    2580 scope.go:117] "RemoveContainer" containerID="a51fe1d28546faf0fe9a07bf1feb8fe79896b42a9a3919053dad2cd4b3cc0d4c"
Sep 08 20:51:18.629762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.629736    2580 generic.go:356] "Generic (PLEG): container finished" podID="abce160c-cd67-4c0d-8a78-19268ae612f5" containerID="212b8383a2a5206b7d12d67ef4121bb7b4685b5e69f6c19dd8dc2afd10ebcb83" exitCode=0
Sep 08 20:51:18.629870 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.629757    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz" event={"ID":"abce160c-cd67-4c0d-8a78-19268ae612f5","Type":"ContainerDied","Data":"212b8383a2a5206b7d12d67ef4121bb7b4685b5e69f6c19dd8dc2afd10ebcb83"}
Sep 08 20:51:18.638422 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.638407    2580 scope.go:117] "RemoveContainer" containerID="a51fe1d28546faf0fe9a07bf1feb8fe79896b42a9a3919053dad2cd4b3cc0d4c"
Sep 08 20:51:18.638687 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:51:18.638668    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a51fe1d28546faf0fe9a07bf1feb8fe79896b42a9a3919053dad2cd4b3cc0d4c\": container with ID starting with a51fe1d28546faf0fe9a07bf1feb8fe79896b42a9a3919053dad2cd4b3cc0d4c not found: ID does not exist" containerID="a51fe1d28546faf0fe9a07bf1feb8fe79896b42a9a3919053dad2cd4b3cc0d4c"
Sep 08 20:51:18.638749 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.638693    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a51fe1d28546faf0fe9a07bf1feb8fe79896b42a9a3919053dad2cd4b3cc0d4c"} err="failed to get container status \"a51fe1d28546faf0fe9a07bf1feb8fe79896b42a9a3919053dad2cd4b3cc0d4c\": rpc error: code = NotFound desc = could not find container \"a51fe1d28546faf0fe9a07bf1feb8fe79896b42a9a3919053dad2cd4b3cc0d4c\": container with ID starting with a51fe1d28546faf0fe9a07bf1feb8fe79896b42a9a3919053dad2cd4b3cc0d4c not found: ID does not exist"
Sep 08 20:51:18.650021 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.649995    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"]
Sep 08 20:51:18.654303 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.654279    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-78dd9f/rate-test-model-78dd9f-kserve-58bf99dfdc-7q2d2"]
Sep 08 20:51:18.751659 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.751596    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:51:18.852305 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.852269    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-model-cache\") pod \"abce160c-cd67-4c0d-8a78-19268ae612f5\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") "
Sep 08 20:51:18.852476 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.852351    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-home\") pod \"abce160c-cd67-4c0d-8a78-19268ae612f5\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") "
Sep 08 20:51:18.852476 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.852378    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r785j\" (UniqueName: \"kubernetes.io/projected/abce160c-cd67-4c0d-8a78-19268ae612f5-kube-api-access-r785j\") pod \"abce160c-cd67-4c0d-8a78-19268ae612f5\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") "
Sep 08 20:51:18.852476 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.852402    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-dshm\") pod \"abce160c-cd67-4c0d-8a78-19268ae612f5\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") "
Sep 08 20:51:18.852476 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.852433    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abce160c-cd67-4c0d-8a78-19268ae612f5-tls-certs\") pod \"abce160c-cd67-4c0d-8a78-19268ae612f5\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") "
Sep 08 20:51:18.852704 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.852558    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-model-cache" (OuterVolumeSpecName: "model-cache") pod "abce160c-cd67-4c0d-8a78-19268ae612f5" (UID: "abce160c-cd67-4c0d-8a78-19268ae612f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:51:18.852704 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.852567    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-tmp-dir\") pod \"abce160c-cd67-4c0d-8a78-19268ae612f5\" (UID: \"abce160c-cd67-4c0d-8a78-19268ae612f5\") "
Sep 08 20:51:18.852704 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.852671    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-home" (OuterVolumeSpecName: "home") pod "abce160c-cd67-4c0d-8a78-19268ae612f5" (UID: "abce160c-cd67-4c0d-8a78-19268ae612f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:51:18.852860 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.852785    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "abce160c-cd67-4c0d-8a78-19268ae612f5" (UID: "abce160c-cd67-4c0d-8a78-19268ae612f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:51:18.853041 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.853021    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-home\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:51:18.853041 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.853041    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-tmp-dir\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:51:18.853175 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.853051    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-model-cache\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:51:18.854640 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.854617    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-dshm" (OuterVolumeSpecName: "dshm") pod "abce160c-cd67-4c0d-8a78-19268ae612f5" (UID: "abce160c-cd67-4c0d-8a78-19268ae612f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:51:18.854737 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.854644    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abce160c-cd67-4c0d-8a78-19268ae612f5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "abce160c-cd67-4c0d-8a78-19268ae612f5" (UID: "abce160c-cd67-4c0d-8a78-19268ae612f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:51:18.854737 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.854698    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abce160c-cd67-4c0d-8a78-19268ae612f5-kube-api-access-r785j" (OuterVolumeSpecName: "kube-api-access-r785j") pod "abce160c-cd67-4c0d-8a78-19268ae612f5" (UID: "abce160c-cd67-4c0d-8a78-19268ae612f5"). InnerVolumeSpecName "kube-api-access-r785j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:51:18.923950 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.923916    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b5708e0-de84-4800-a8d7-dffa2008d61e" path="/var/lib/kubelet/pods/8b5708e0-de84-4800-a8d7-dffa2008d61e/volumes"
Sep 08 20:51:18.954086 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.954053    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r785j\" (UniqueName: \"kubernetes.io/projected/abce160c-cd67-4c0d-8a78-19268ae612f5-kube-api-access-r785j\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:51:18.954086 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.954081    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abce160c-cd67-4c0d-8a78-19268ae612f5-dshm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:51:18.954263 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:18.954094    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abce160c-cd67-4c0d-8a78-19268ae612f5-tls-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:51:19.637335 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:19.637301    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"
Sep 08 20:51:19.637798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:19.637302    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz" event={"ID":"abce160c-cd67-4c0d-8a78-19268ae612f5","Type":"ContainerDied","Data":"cedf1ffdb2f9bf65e5190fc5997b1a607607434b99480cc97bd75843054ffed6"}
Sep 08 20:51:19.637798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:19.637429    2580 scope.go:117] "RemoveContainer" containerID="212b8383a2a5206b7d12d67ef4121bb7b4685b5e69f6c19dd8dc2afd10ebcb83"
Sep 08 20:51:19.653652 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:19.653627    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"]
Sep 08 20:51:19.657534 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:19.657510    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-c9590b/rate-test-model-c9590b-kserve-65b689874c-fbdlz"]
Sep 08 20:51:20.923737 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:20.923703    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abce160c-cd67-4c0d-8a78-19268ae612f5" path="/var/lib/kubelet/pods/abce160c-cd67-4c0d-8a78-19268ae612f5/volumes"
Sep 08 20:51:27.091122 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:27.091087    2580 scope.go:117] "RemoveContainer" containerID="9a08a8a9d6cdd86680a7bc8f6633da7f8c8b85a414ed3c7923d6dcd685442c67"
Sep 08 20:51:32.668062 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:32.668019    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:34.302753 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:34.302713    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:37.041501 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:37.041469    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:37.134408 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:37.134350    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:38.103337 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.103303    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"]
Sep 08 20:51:38.103798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.103722    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e28c4038-41cf-46fd-91e3-410d097ca036" containerName="istio-proxy"
Sep 08 20:51:38.103798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.103735    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e28c4038-41cf-46fd-91e3-410d097ca036" containerName="istio-proxy"
Sep 08 20:51:38.103798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.103743    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07a0bd22-9061-4fcd-9e3f-0e4aa48af090" containerName="istio-proxy"
Sep 08 20:51:38.103798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.103748    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="07a0bd22-9061-4fcd-9e3f-0e4aa48af090" containerName="istio-proxy"
Sep 08 20:51:38.103798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.103754    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abce160c-cd67-4c0d-8a78-19268ae612f5" containerName="main"
Sep 08 20:51:38.103798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.103760    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="abce160c-cd67-4c0d-8a78-19268ae612f5" containerName="main"
Sep 08 20:51:38.103798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.103769    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b5708e0-de84-4800-a8d7-dffa2008d61e" containerName="main"
Sep 08 20:51:38.103798 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.103774    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b5708e0-de84-4800-a8d7-dffa2008d61e" containerName="main"
Sep 08 20:51:38.104079 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.103844    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e28c4038-41cf-46fd-91e3-410d097ca036" containerName="istio-proxy"
Sep 08 20:51:38.104079 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.103853    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e28c4038-41cf-46fd-91e3-410d097ca036" containerName="istio-proxy"
Sep 08 20:51:38.104079 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.103859    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="abce160c-cd67-4c0d-8a78-19268ae612f5" containerName="main"
Sep 08 20:51:38.104079 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.103865    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b5708e0-de84-4800-a8d7-dffa2008d61e" containerName="main"
Sep 08 20:51:38.107075 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.107054    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:51:38.109223 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.109202    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-0566ff-serving-cert\""
Sep 08 20:51:38.117609 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.117578    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"]
Sep 08 20:51:38.140708 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.140674    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:38.204739 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.204685    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:38.244154 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.244114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs5mp\" (UniqueName: \"kubernetes.io/projected/68da7896-f3c6-4826-b385-d5c90591a37e-kube-api-access-xs5mp\") pod \"maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8\" (UID: \"68da7896-f3c6-4826-b385-d5c90591a37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:51:38.244327 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.244234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68da7896-f3c6-4826-b385-d5c90591a37e-metrics-certs\") pod \"maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8\" (UID: \"68da7896-f3c6-4826-b385-d5c90591a37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:51:38.244327 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.244270    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68da7896-f3c6-4826-b385-d5c90591a37e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8\" (UID: \"68da7896-f3c6-4826-b385-d5c90591a37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:51:38.244327 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.244293    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68da7896-f3c6-4826-b385-d5c90591a37e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8\" (UID: \"68da7896-f3c6-4826-b385-d5c90591a37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:51:38.310224 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.310197    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:38.345683 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.345647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68da7896-f3c6-4826-b385-d5c90591a37e-metrics-certs\") pod \"maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8\" (UID: \"68da7896-f3c6-4826-b385-d5c90591a37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:51:38.345864 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.345690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68da7896-f3c6-4826-b385-d5c90591a37e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8\" (UID: \"68da7896-f3c6-4826-b385-d5c90591a37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:51:38.345864 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.345714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68da7896-f3c6-4826-b385-d5c90591a37e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8\" (UID: \"68da7896-f3c6-4826-b385-d5c90591a37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:51:38.345864 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.345776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs5mp\" (UniqueName: \"kubernetes.io/projected/68da7896-f3c6-4826-b385-d5c90591a37e-kube-api-access-xs5mp\") pod \"maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8\" (UID: \"68da7896-f3c6-4826-b385-d5c90591a37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:51:38.346477 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.346454    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68da7896-f3c6-4826-b385-d5c90591a37e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8\" (UID: \"68da7896-f3c6-4826-b385-d5c90591a37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:51:38.348187 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.348166    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68da7896-f3c6-4826-b385-d5c90591a37e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8\" (UID: \"68da7896-f3c6-4826-b385-d5c90591a37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:51:38.348280 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.348211    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68da7896-f3c6-4826-b385-d5c90591a37e-metrics-certs\") pod \"maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8\" (UID: \"68da7896-f3c6-4826-b385-d5c90591a37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:51:38.354036 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.353987    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs5mp\" (UniqueName: \"kubernetes.io/projected/68da7896-f3c6-4826-b385-d5c90591a37e-kube-api-access-xs5mp\") pod \"maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8\" (UID: \"68da7896-f3c6-4826-b385-d5c90591a37e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:51:38.420139 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.420103    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:51:38.600476 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.600433    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:38.695332 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.695301    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"]
Sep 08 20:51:38.697387 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:51:38.697356    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68da7896_f3c6_4826_b385_d5c90591a37e.slice/crio-db5b5263d924c8a140e5dd1ae6d17fdd591e0e3ead85b134067044e851a2a0b0 WatchSource:0}: Error finding container db5b5263d924c8a140e5dd1ae6d17fdd591e0e3ead85b134067044e851a2a0b0: Status 404 returned error can't find the container with id db5b5263d924c8a140e5dd1ae6d17fdd591e0e3ead85b134067044e851a2a0b0
Sep 08 20:51:38.698422 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.698394    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:51:38.698762 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.698746    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 20:51:38.737024 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:38.736987    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8" event={"ID":"68da7896-f3c6-4826-b385-d5c90591a37e","Type":"ContainerStarted","Data":"db5b5263d924c8a140e5dd1ae6d17fdd591e0e3ead85b134067044e851a2a0b0"}
Sep 08 20:51:39.744921 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:39.744883    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8" event={"ID":"68da7896-f3c6-4826-b385-d5c90591a37e","Type":"ContainerStarted","Data":"8f9d7bb4cf0c0fd4cf5cce7a8b8f70e701f5e4bd9a37191f88541ecb399568ad"}
Sep 08 20:51:39.745335 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:39.745030    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:51:39.761048 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:39.760990    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8" podStartSLOduration=1.450312179 podStartE2EDuration="1.76094377s" totalImagesPullingTime="310.631591ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:51:39.760293047 +0000 UTC m=+1573.307542521" watchObservedRunningTime="2026-09-08 20:51:39.76094377 +0000 UTC m=+1573.308193264"
Sep 08 20:51:45.755647 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:51:45.755614    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:52:01.927608 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:01.927567    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:52:12.402324 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.402284    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"]
Sep 08 20:52:12.402754 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.402738    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e28c4038-41cf-46fd-91e3-410d097ca036" containerName="istio-proxy"
Sep 08 20:52:12.402754 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.402755    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e28c4038-41cf-46fd-91e3-410d097ca036" containerName="istio-proxy"
Sep 08 20:52:12.406031 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.406009    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.408686 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.408662    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-aab326\"/\"ipp-route-aab326-kserve-self-signed-certs\""
Sep 08 20:52:12.409468 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.409445    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-aab326\"/\"kube-root-ca.crt\""
Sep 08 20:52:12.409576 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.409445    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-aab326\"/\"openshift-service-ca.crt\""
Sep 08 20:52:12.409576 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.409501    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-aab326\"/\"default-dockercfg-9qwwq\""
Sep 08 20:52:12.416898 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.416876    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"]
Sep 08 20:52:12.488004 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.487939    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-model-cache\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.488175 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.488025    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-home\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.488175 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.488095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36f9914-fd54-42ce-a2a0-9b9322afac30-tls-certs\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.488175 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.488138    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b82n2\" (UniqueName: \"kubernetes.io/projected/e36f9914-fd54-42ce-a2a0-9b9322afac30-kube-api-access-b82n2\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.488350 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.488199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-tmp-dir\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.488350 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.488222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-dshm\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.490792 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.490751    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:52:12.588751 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.588716    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36f9914-fd54-42ce-a2a0-9b9322afac30-tls-certs\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.588751 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.588768    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b82n2\" (UniqueName: \"kubernetes.io/projected/e36f9914-fd54-42ce-a2a0-9b9322afac30-kube-api-access-b82n2\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.589053 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.588806    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-tmp-dir\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.589053 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.588831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-dshm\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.589053 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.588852    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-model-cache\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.589053 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.588876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-home\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.589348 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.589328    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-home\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.589422 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.589332    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-tmp-dir\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.589422 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.589374    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-model-cache\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.591151 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.591130    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-dshm\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.591295 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.591275    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36f9914-fd54-42ce-a2a0-9b9322afac30-tls-certs\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.597379 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.597357    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b82n2\" (UniqueName: \"kubernetes.io/projected/e36f9914-fd54-42ce-a2a0-9b9322afac30-kube-api-access-b82n2\") pod \"ipp-route-aab326-kserve-5575986bc-zt85q\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") " pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.717285 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.717196    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:12.988742 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:12.988715    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"]
Sep 08 20:52:12.990453 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:52:12.990423    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode36f9914_fd54_42ce_a2a0_9b9322afac30.slice/crio-bbca3cef7c89a455a84e612859a1fb3985a8597deaae7eb338cf9edb7393c823 WatchSource:0}: Error finding container bbca3cef7c89a455a84e612859a1fb3985a8597deaae7eb338cf9edb7393c823: Status 404 returned error can't find the container with id bbca3cef7c89a455a84e612859a1fb3985a8597deaae7eb338cf9edb7393c823
Sep 08 20:52:13.490614 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:13.490579    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:52:13.919884 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:13.919839    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q" event={"ID":"e36f9914-fd54-42ce-a2a0-9b9322afac30","Type":"ContainerStarted","Data":"d1f1bca59a6e55977b3c22aa82dd1cf5f5cb6eb8b5d6376b130d2fcc890e10ce"}
Sep 08 20:52:13.919884 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:13.919889    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q" event={"ID":"e36f9914-fd54-42ce-a2a0-9b9322afac30","Type":"ContainerStarted","Data":"bbca3cef7c89a455a84e612859a1fb3985a8597deaae7eb338cf9edb7393c823"}
Sep 08 20:52:13.937718 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:13.937663    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q" podStartSLOduration=1.937648695 podStartE2EDuration="1.937648695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:52:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:52:13.936153964 +0000 UTC m=+1607.483403445" watchObservedRunningTime="2026-09-08 20:52:13.937648695 +0000 UTC m=+1607.484898176"
Sep 08 20:52:17.429291 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:17.429256    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:52:17.527889 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:17.527854    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:52:22.717426 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:22.717344    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:22.717426 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:22.717409    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:22.730480 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:22.730447    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:22.976571 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:22.976471    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:52:43.290909 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:43.290876    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:52:43.640835 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:43.640806    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:52:43.867264 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:43.867229    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:52:46.374640 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:46.374588    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:52:46.472166 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:46.472130    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:52:47.693869 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:47.693837    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:52:47.796818 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:47.796785    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:52:48.060250 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:48.060164    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:52:48.171056 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:48.171018    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:52:48.269466 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:52:48.269427    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:11.346433 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:11.346396    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:18.200902 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:18.200868    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:18.301948 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:18.301912    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:18.400983 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:18.400924    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:18.504099 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:18.504005    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:18.761848 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:18.761746    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:25.074024 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:25.073980    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:25.185105 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:25.185068    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:25.367788 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:25.367701    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:29.455633 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:29.455592    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:44.726640 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:44.726609    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:44.825519 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:44.825482    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:44.924263 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:44.924231    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:45.015483 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:45.015399    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:45.488072 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:45.488038    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:50.360116 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:50.360077    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"]
Sep 08 20:53:50.456691 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:50.456653    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:55.432384 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:55.432350    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:53:55.521076 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:53:55.521030    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:05.379825 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.379783    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q" podUID="e36f9914-fd54-42ce-a2a0-9b9322afac30" containerName="main" containerID="cri-o://d1f1bca59a6e55977b3c22aa82dd1cf5f5cb6eb8b5d6376b130d2fcc890e10ce" gracePeriod=45
Sep 08 20:54:05.636028 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.635939    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:54:05.675171 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.675135    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-tmp-dir\") pod \"e36f9914-fd54-42ce-a2a0-9b9322afac30\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") "
Sep 08 20:54:05.675342 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.675220    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-model-cache\") pod \"e36f9914-fd54-42ce-a2a0-9b9322afac30\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") "
Sep 08 20:54:05.675342 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.675287    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-dshm\") pod \"e36f9914-fd54-42ce-a2a0-9b9322afac30\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") "
Sep 08 20:54:05.675425 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.675358    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-home\") pod \"e36f9914-fd54-42ce-a2a0-9b9322afac30\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") "
Sep 08 20:54:05.675425 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.675403    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36f9914-fd54-42ce-a2a0-9b9322afac30-tls-certs\") pod \"e36f9914-fd54-42ce-a2a0-9b9322afac30\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") "
Sep 08 20:54:05.675505 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.675427    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b82n2\" (UniqueName: \"kubernetes.io/projected/e36f9914-fd54-42ce-a2a0-9b9322afac30-kube-api-access-b82n2\") pod \"e36f9914-fd54-42ce-a2a0-9b9322afac30\" (UID: \"e36f9914-fd54-42ce-a2a0-9b9322afac30\") "
Sep 08 20:54:05.675505 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.675422    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e36f9914-fd54-42ce-a2a0-9b9322afac30" (UID: "e36f9914-fd54-42ce-a2a0-9b9322afac30"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:54:05.675505 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.675473    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-model-cache" (OuterVolumeSpecName: "model-cache") pod "e36f9914-fd54-42ce-a2a0-9b9322afac30" (UID: "e36f9914-fd54-42ce-a2a0-9b9322afac30"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:54:05.675754 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.675727    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-home" (OuterVolumeSpecName: "home") pod "e36f9914-fd54-42ce-a2a0-9b9322afac30" (UID: "e36f9914-fd54-42ce-a2a0-9b9322afac30"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:54:05.675870 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.675855    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-home\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:54:05.675934 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.675872    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-tmp-dir\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:54:05.675934 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.675881    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-model-cache\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:54:05.677914 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.677890    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-dshm" (OuterVolumeSpecName: "dshm") pod "e36f9914-fd54-42ce-a2a0-9b9322afac30" (UID: "e36f9914-fd54-42ce-a2a0-9b9322afac30"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:54:05.678192 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.678163    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e36f9914-fd54-42ce-a2a0-9b9322afac30-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e36f9914-fd54-42ce-a2a0-9b9322afac30" (UID: "e36f9914-fd54-42ce-a2a0-9b9322afac30"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:54:05.678402 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.678381    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e36f9914-fd54-42ce-a2a0-9b9322afac30-kube-api-access-b82n2" (OuterVolumeSpecName: "kube-api-access-b82n2") pod "e36f9914-fd54-42ce-a2a0-9b9322afac30" (UID: "e36f9914-fd54-42ce-a2a0-9b9322afac30"). InnerVolumeSpecName "kube-api-access-b82n2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:54:05.777063 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.777030    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e36f9914-fd54-42ce-a2a0-9b9322afac30-dshm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:54:05.777063 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.777058    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e36f9914-fd54-42ce-a2a0-9b9322afac30-tls-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:54:05.777063 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:05.777071    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b82n2\" (UniqueName: \"kubernetes.io/projected/e36f9914-fd54-42ce-a2a0-9b9322afac30-kube-api-access-b82n2\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:54:06.490484 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:06.490445    2580 generic.go:356] "Generic (PLEG): container finished" podID="e36f9914-fd54-42ce-a2a0-9b9322afac30" containerID="d1f1bca59a6e55977b3c22aa82dd1cf5f5cb6eb8b5d6376b130d2fcc890e10ce" exitCode=0
Sep 08 20:54:06.490484 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:06.490489    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"
Sep 08 20:54:06.491021 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:06.490525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q" event={"ID":"e36f9914-fd54-42ce-a2a0-9b9322afac30","Type":"ContainerDied","Data":"d1f1bca59a6e55977b3c22aa82dd1cf5f5cb6eb8b5d6376b130d2fcc890e10ce"}
Sep 08 20:54:06.491021 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:06.490570    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q" event={"ID":"e36f9914-fd54-42ce-a2a0-9b9322afac30","Type":"ContainerDied","Data":"bbca3cef7c89a455a84e612859a1fb3985a8597deaae7eb338cf9edb7393c823"}
Sep 08 20:54:06.491021 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:06.490589    2580 scope.go:117] "RemoveContainer" containerID="d1f1bca59a6e55977b3c22aa82dd1cf5f5cb6eb8b5d6376b130d2fcc890e10ce"
Sep 08 20:54:06.500440 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:06.500421    2580 scope.go:117] "RemoveContainer" containerID="d1f1bca59a6e55977b3c22aa82dd1cf5f5cb6eb8b5d6376b130d2fcc890e10ce"
Sep 08 20:54:06.500711 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:54:06.500692    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1f1bca59a6e55977b3c22aa82dd1cf5f5cb6eb8b5d6376b130d2fcc890e10ce\": container with ID starting with d1f1bca59a6e55977b3c22aa82dd1cf5f5cb6eb8b5d6376b130d2fcc890e10ce not found: ID does not exist" containerID="d1f1bca59a6e55977b3c22aa82dd1cf5f5cb6eb8b5d6376b130d2fcc890e10ce"
Sep 08 20:54:06.500756 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:06.500722    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1f1bca59a6e55977b3c22aa82dd1cf5f5cb6eb8b5d6376b130d2fcc890e10ce"} err="failed to get container status \"d1f1bca59a6e55977b3c22aa82dd1cf5f5cb6eb8b5d6376b130d2fcc890e10ce\": rpc error: code = NotFound desc = could not find container \"d1f1bca59a6e55977b3c22aa82dd1cf5f5cb6eb8b5d6376b130d2fcc890e10ce\": container with ID starting with d1f1bca59a6e55977b3c22aa82dd1cf5f5cb6eb8b5d6376b130d2fcc890e10ce not found: ID does not exist"
Sep 08 20:54:06.511874 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:06.511847    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"]
Sep 08 20:54:06.515252 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:06.515226    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-aab326/ipp-route-aab326-kserve-5575986bc-zt85q"]
Sep 08 20:54:06.924665 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:06.924631    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e36f9914-fd54-42ce-a2a0-9b9322afac30" path="/var/lib/kubelet/pods/e36f9914-fd54-42ce-a2a0-9b9322afac30/volumes"
Sep 08 20:54:09.696429 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:09.696399    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:26.839550 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:26.839517    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:27.949496 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:27.949464    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:33.740871 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:33.740822    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:33.845297 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:33.845263    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:34.082559 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:34.082483    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:34.767929 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:34.767896    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:38.448743 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:38.448705    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:39.718610 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:39.718571    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:44.761753 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:44.761720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:45.631937 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:45.631892    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"]
Sep 08 20:54:45.632173 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:45.632123    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8" podUID="68da7896-f3c6-4826-b385-d5c90591a37e" containerName="maas-api" containerID="cri-o://8f9d7bb4cf0c0fd4cf5cce7a8b8f70e701f5e4bd9a37191f88541ecb399568ad" gracePeriod=30
Sep 08 20:54:45.680655 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:45.680603    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:45.750449 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:45.750377    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8" podUID="68da7896-f3c6-4826-b385-d5c90591a37e" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.65:8443/health\": dial tcp 10.133.0.65:8443: connect: connection refused"
Sep 08 20:54:45.828727 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:45.828688    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:45.885942 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:45.885869    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:54:45.969093 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:45.969061    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:45.971170 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:45.971148    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68da7896-f3c6-4826-b385-d5c90591a37e-openshift-service-ca\") pod \"68da7896-f3c6-4826-b385-d5c90591a37e\" (UID: \"68da7896-f3c6-4826-b385-d5c90591a37e\") "
Sep 08 20:54:45.971226 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:45.971206    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68da7896-f3c6-4826-b385-d5c90591a37e-maas-api-tls\") pod \"68da7896-f3c6-4826-b385-d5c90591a37e\" (UID: \"68da7896-f3c6-4826-b385-d5c90591a37e\") "
Sep 08 20:54:45.971261 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:45.971233    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68da7896-f3c6-4826-b385-d5c90591a37e-metrics-certs\") pod \"68da7896-f3c6-4826-b385-d5c90591a37e\" (UID: \"68da7896-f3c6-4826-b385-d5c90591a37e\") "
Sep 08 20:54:45.971295 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:45.971263    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xs5mp\" (UniqueName: \"kubernetes.io/projected/68da7896-f3c6-4826-b385-d5c90591a37e-kube-api-access-xs5mp\") pod \"68da7896-f3c6-4826-b385-d5c90591a37e\" (UID: \"68da7896-f3c6-4826-b385-d5c90591a37e\") "
Sep 08 20:54:45.971564 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:45.971540    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68da7896-f3c6-4826-b385-d5c90591a37e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "68da7896-f3c6-4826-b385-d5c90591a37e" (UID: "68da7896-f3c6-4826-b385-d5c90591a37e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:54:45.973408 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:45.973383    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68da7896-f3c6-4826-b385-d5c90591a37e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "68da7896-f3c6-4826-b385-d5c90591a37e" (UID: "68da7896-f3c6-4826-b385-d5c90591a37e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:54:45.973408 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:45.973412    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68da7896-f3c6-4826-b385-d5c90591a37e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "68da7896-f3c6-4826-b385-d5c90591a37e" (UID: "68da7896-f3c6-4826-b385-d5c90591a37e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:54:45.973553 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:45.973486    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68da7896-f3c6-4826-b385-d5c90591a37e-kube-api-access-xs5mp" (OuterVolumeSpecName: "kube-api-access-xs5mp") pod "68da7896-f3c6-4826-b385-d5c90591a37e" (UID: "68da7896-f3c6-4826-b385-d5c90591a37e"). InnerVolumeSpecName "kube-api-access-xs5mp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:54:46.072887 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.072845    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68da7896-f3c6-4826-b385-d5c90591a37e-openshift-service-ca\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:54:46.072887 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.072879    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68da7896-f3c6-4826-b385-d5c90591a37e-maas-api-tls\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:54:46.072887 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.072890    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68da7896-f3c6-4826-b385-d5c90591a37e-metrics-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:54:46.072887 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.072900    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xs5mp\" (UniqueName: \"kubernetes.io/projected/68da7896-f3c6-4826-b385-d5c90591a37e-kube-api-access-xs5mp\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:54:46.077701 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.077673    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:46.250642 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.250562    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:46.562101 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.561953    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:46.691602 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.691572    2580 generic.go:356] "Generic (PLEG): container finished" podID="68da7896-f3c6-4826-b385-d5c90591a37e" containerID="8f9d7bb4cf0c0fd4cf5cce7a8b8f70e701f5e4bd9a37191f88541ecb399568ad" exitCode=0
Sep 08 20:54:46.691787 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.691622    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"
Sep 08 20:54:46.691787 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.691665    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8" event={"ID":"68da7896-f3c6-4826-b385-d5c90591a37e","Type":"ContainerDied","Data":"8f9d7bb4cf0c0fd4cf5cce7a8b8f70e701f5e4bd9a37191f88541ecb399568ad"}
Sep 08 20:54:46.691787 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.691718    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8" event={"ID":"68da7896-f3c6-4826-b385-d5c90591a37e","Type":"ContainerDied","Data":"db5b5263d924c8a140e5dd1ae6d17fdd591e0e3ead85b134067044e851a2a0b0"}
Sep 08 20:54:46.691787 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.691742    2580 scope.go:117] "RemoveContainer" containerID="8f9d7bb4cf0c0fd4cf5cce7a8b8f70e701f5e4bd9a37191f88541ecb399568ad"
Sep 08 20:54:46.701227 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.701206    2580 scope.go:117] "RemoveContainer" containerID="8f9d7bb4cf0c0fd4cf5cce7a8b8f70e701f5e4bd9a37191f88541ecb399568ad"
Sep 08 20:54:46.701542 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:54:46.701520    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f9d7bb4cf0c0fd4cf5cce7a8b8f70e701f5e4bd9a37191f88541ecb399568ad\": container with ID starting with 8f9d7bb4cf0c0fd4cf5cce7a8b8f70e701f5e4bd9a37191f88541ecb399568ad not found: ID does not exist" containerID="8f9d7bb4cf0c0fd4cf5cce7a8b8f70e701f5e4bd9a37191f88541ecb399568ad"
Sep 08 20:54:46.701599 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.701554    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f9d7bb4cf0c0fd4cf5cce7a8b8f70e701f5e4bd9a37191f88541ecb399568ad"} err="failed to get container status \"8f9d7bb4cf0c0fd4cf5cce7a8b8f70e701f5e4bd9a37191f88541ecb399568ad\": rpc error: code = NotFound desc = could not find container \"8f9d7bb4cf0c0fd4cf5cce7a8b8f70e701f5e4bd9a37191f88541ecb399568ad\": container with ID starting with 8f9d7bb4cf0c0fd4cf5cce7a8b8f70e701f5e4bd9a37191f88541ecb399568ad not found: ID does not exist"
Sep 08 20:54:46.715755 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.715728    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"]
Sep 08 20:54:46.718172 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.718149    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0566ff-674cd8c66b-m77b8"]
Sep 08 20:54:46.923833 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:46.923798    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68da7896-f3c6-4826-b385-d5c90591a37e" path="/var/lib/kubelet/pods/68da7896-f3c6-4826-b385-d5c90591a37e/volumes"
Sep 08 20:54:48.915672 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:48.915635    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:53.067189 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:53.067153    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:53.158793 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:53.158759    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:53.461784 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:53.461749    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:53.966274 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:53.966243    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:54:58.682208 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:54:58.682167    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:04.349803 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.349766    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"]
Sep 08 20:55:04.350300 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.350072    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb" podUID="124d18f2-8058-4281-8d27-748adbd924ab" containerName="maas-api" containerID="cri-o://022ca8cd7a83ea775b255267b92d79ef21d7a64db9ec9c0a23790e6971530f3e" gracePeriod=30
Sep 08 20:55:04.386139 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.386110    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:04.484032 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.483994    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:04.588188 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.588155    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:04.605047 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.604978    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:55:04.651738 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.651696    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/124d18f2-8058-4281-8d27-748adbd924ab-metrics-certs\") pod \"124d18f2-8058-4281-8d27-748adbd924ab\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") "
Sep 08 20:55:04.651941 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.651795    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/124d18f2-8058-4281-8d27-748adbd924ab-openshift-service-ca\") pod \"124d18f2-8058-4281-8d27-748adbd924ab\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") "
Sep 08 20:55:04.651941 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.651851    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94ng5\" (UniqueName: \"kubernetes.io/projected/124d18f2-8058-4281-8d27-748adbd924ab-kube-api-access-94ng5\") pod \"124d18f2-8058-4281-8d27-748adbd924ab\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") "
Sep 08 20:55:04.651941 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.651913    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/124d18f2-8058-4281-8d27-748adbd924ab-maas-api-tls\") pod \"124d18f2-8058-4281-8d27-748adbd924ab\" (UID: \"124d18f2-8058-4281-8d27-748adbd924ab\") "
Sep 08 20:55:04.652310 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.652263    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/124d18f2-8058-4281-8d27-748adbd924ab-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "124d18f2-8058-4281-8d27-748adbd924ab" (UID: "124d18f2-8058-4281-8d27-748adbd924ab"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:55:04.654166 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.654125    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/124d18f2-8058-4281-8d27-748adbd924ab-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "124d18f2-8058-4281-8d27-748adbd924ab" (UID: "124d18f2-8058-4281-8d27-748adbd924ab"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:55:04.654292 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.654227    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/124d18f2-8058-4281-8d27-748adbd924ab-kube-api-access-94ng5" (OuterVolumeSpecName: "kube-api-access-94ng5") pod "124d18f2-8058-4281-8d27-748adbd924ab" (UID: "124d18f2-8058-4281-8d27-748adbd924ab"). InnerVolumeSpecName "kube-api-access-94ng5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:55:04.654487 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.654470    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/124d18f2-8058-4281-8d27-748adbd924ab-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "124d18f2-8058-4281-8d27-748adbd924ab" (UID: "124d18f2-8058-4281-8d27-748adbd924ab"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:55:04.728799 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.728766    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:04.753468 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.753434    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/124d18f2-8058-4281-8d27-748adbd924ab-maas-api-tls\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:04.753468 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.753465    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/124d18f2-8058-4281-8d27-748adbd924ab-metrics-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:04.753468 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.753478    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/124d18f2-8058-4281-8d27-748adbd924ab-openshift-service-ca\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:04.753705 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.753490    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94ng5\" (UniqueName: \"kubernetes.io/projected/124d18f2-8058-4281-8d27-748adbd924ab-kube-api-access-94ng5\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:04.782588 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.782554    2580 generic.go:356] "Generic (PLEG): container finished" podID="124d18f2-8058-4281-8d27-748adbd924ab" containerID="022ca8cd7a83ea775b255267b92d79ef21d7a64db9ec9c0a23790e6971530f3e" exitCode=0
Sep 08 20:55:04.782761 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.782606    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"
Sep 08 20:55:04.782761 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.782630    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb" event={"ID":"124d18f2-8058-4281-8d27-748adbd924ab","Type":"ContainerDied","Data":"022ca8cd7a83ea775b255267b92d79ef21d7a64db9ec9c0a23790e6971530f3e"}
Sep 08 20:55:04.782761 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.782681    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb" event={"ID":"124d18f2-8058-4281-8d27-748adbd924ab","Type":"ContainerDied","Data":"1ffe37c9186b93aa09478340cbefc96db4d5719a66394d07db060965b7c30432"}
Sep 08 20:55:04.782761 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.782703    2580 scope.go:117] "RemoveContainer" containerID="022ca8cd7a83ea775b255267b92d79ef21d7a64db9ec9c0a23790e6971530f3e"
Sep 08 20:55:04.795117 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.795091    2580 scope.go:117] "RemoveContainer" containerID="022ca8cd7a83ea775b255267b92d79ef21d7a64db9ec9c0a23790e6971530f3e"
Sep 08 20:55:04.795527 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:55:04.795506    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"022ca8cd7a83ea775b255267b92d79ef21d7a64db9ec9c0a23790e6971530f3e\": container with ID starting with 022ca8cd7a83ea775b255267b92d79ef21d7a64db9ec9c0a23790e6971530f3e not found: ID does not exist" containerID="022ca8cd7a83ea775b255267b92d79ef21d7a64db9ec9c0a23790e6971530f3e"
Sep 08 20:55:04.795601 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.795539    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"022ca8cd7a83ea775b255267b92d79ef21d7a64db9ec9c0a23790e6971530f3e"} err="failed to get container status \"022ca8cd7a83ea775b255267b92d79ef21d7a64db9ec9c0a23790e6971530f3e\": rpc error: code = NotFound desc = could not find container \"022ca8cd7a83ea775b255267b92d79ef21d7a64db9ec9c0a23790e6971530f3e\": container with ID starting with 022ca8cd7a83ea775b255267b92d79ef21d7a64db9ec9c0a23790e6971530f3e not found: ID does not exist"
Sep 08 20:55:04.806497 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.806466    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"]
Sep 08 20:55:04.816104 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.816073    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-78dd9f-68d698fc59-c48tb"]
Sep 08 20:55:04.924581 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:04.924551    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="124d18f2-8058-4281-8d27-748adbd924ab" path="/var/lib/kubelet/pods/124d18f2-8058-4281-8d27-748adbd924ab/volumes"
Sep 08 20:55:05.101800 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:05.101759    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:11.859377 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:11.859343    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:11.962129 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:11.962098    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:12.138073 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:12.138040    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"]
Sep 08 20:55:12.138277 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:12.138256    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h" podUID="fe4d6ee4-d36e-4929-b356-15fde3dde9aa" containerName="istio-proxy" containerID="cri-o://4158ddb88378c0c6a0a213c7a6a66bfacbc34560966f12e812d14ef4d83e927f" gracePeriod=30
Sep 08 20:55:12.163892 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:12.163860    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:12.815244 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:12.815213    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:17.400145 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.400118    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:55:17.472260 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.472167    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-workload-certs\") pod \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") "
Sep 08 20:55:17.472260 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.472238    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-envoy\") pod \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") "
Sep 08 20:55:17.472487 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.472262    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-podinfo\") pod \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") "
Sep 08 20:55:17.472487 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.472285    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-credential-socket\") pod \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") "
Sep 08 20:55:17.472487 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.472319    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hzdw\" (UniqueName: \"kubernetes.io/projected/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-kube-api-access-7hzdw\") pod \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") "
Sep 08 20:55:17.472487 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.472356    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istiod-ca-cert\") pod \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") "
Sep 08 20:55:17.472487 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.472390    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-token\") pod \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") "
Sep 08 20:55:17.472487 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.472423    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-workload-socket\") pod \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") "
Sep 08 20:55:17.472487 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.472477    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-data\") pod \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\" (UID: \"fe4d6ee4-d36e-4929-b356-15fde3dde9aa\") "
Sep 08 20:55:17.472846 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.472511    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fe4d6ee4-d36e-4929-b356-15fde3dde9aa" (UID: "fe4d6ee4-d36e-4929-b356-15fde3dde9aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:55:17.472846 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.472619    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fe4d6ee4-d36e-4929-b356-15fde3dde9aa" (UID: "fe4d6ee4-d36e-4929-b356-15fde3dde9aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:55:17.472846 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.472720    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "fe4d6ee4-d36e-4929-b356-15fde3dde9aa" (UID: "fe4d6ee4-d36e-4929-b356-15fde3dde9aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:55:17.472846 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.472810    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fe4d6ee4-d36e-4929-b356-15fde3dde9aa" (UID: "fe4d6ee4-d36e-4929-b356-15fde3dde9aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:55:17.473071 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.472912    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fe4d6ee4-d36e-4929-b356-15fde3dde9aa" (UID: "fe4d6ee4-d36e-4929-b356-15fde3dde9aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:55:17.473071 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.472985    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-credential-socket\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:17.473071 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.473008    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istiod-ca-cert\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:17.473071 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.473021    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-data\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:17.473071 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.473034    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-workload-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:17.474492 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.474466    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fe4d6ee4-d36e-4929-b356-15fde3dde9aa" (UID: "fe4d6ee4-d36e-4929-b356-15fde3dde9aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:55:17.474603 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.474566    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "fe4d6ee4-d36e-4929-b356-15fde3dde9aa" (UID: "fe4d6ee4-d36e-4929-b356-15fde3dde9aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:55:17.474847 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.474820    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-kube-api-access-7hzdw" (OuterVolumeSpecName: "kube-api-access-7hzdw") pod "fe4d6ee4-d36e-4929-b356-15fde3dde9aa" (UID: "fe4d6ee4-d36e-4929-b356-15fde3dde9aa"). InnerVolumeSpecName "kube-api-access-7hzdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:55:17.474847 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.474834    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fe4d6ee4-d36e-4929-b356-15fde3dde9aa" (UID: "fe4d6ee4-d36e-4929-b356-15fde3dde9aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:55:17.573848 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.573810    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-envoy\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:17.573848 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.573839    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-podinfo\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:17.573848 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.573850    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hzdw\" (UniqueName: \"kubernetes.io/projected/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-kube-api-access-7hzdw\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:17.573848 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.573860    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-istio-token\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:17.574161 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.573869    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4d6ee4-d36e-4929-b356-15fde3dde9aa-workload-socket\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:17.847945 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.847852    2580 generic.go:356] "Generic (PLEG): container finished" podID="fe4d6ee4-d36e-4929-b356-15fde3dde9aa" containerID="4158ddb88378c0c6a0a213c7a6a66bfacbc34560966f12e812d14ef4d83e927f" exitCode=0
Sep 08 20:55:17.847945 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.847905    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"
Sep 08 20:55:17.847945 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.847927    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h" event={"ID":"fe4d6ee4-d36e-4929-b356-15fde3dde9aa","Type":"ContainerDied","Data":"4158ddb88378c0c6a0a213c7a6a66bfacbc34560966f12e812d14ef4d83e927f"}
Sep 08 20:55:17.848198 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.847994    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h" event={"ID":"fe4d6ee4-d36e-4929-b356-15fde3dde9aa","Type":"ContainerDied","Data":"6dbee8e9e73cb536d3902774a42f8f0b950cb1fd6266f7eac98ebd0f8f4a5e53"}
Sep 08 20:55:17.848198 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.848017    2580 scope.go:117] "RemoveContainer" containerID="4158ddb88378c0c6a0a213c7a6a66bfacbc34560966f12e812d14ef4d83e927f"
Sep 08 20:55:17.857572 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.857553    2580 scope.go:117] "RemoveContainer" containerID="c45f09717675550df07a3d81dbe1265ef005d0955b8c2604ada5d2e78911ee84"
Sep 08 20:55:17.865560 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.865541    2580 scope.go:117] "RemoveContainer" containerID="4158ddb88378c0c6a0a213c7a6a66bfacbc34560966f12e812d14ef4d83e927f"
Sep 08 20:55:17.865845 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:55:17.865825    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4158ddb88378c0c6a0a213c7a6a66bfacbc34560966f12e812d14ef4d83e927f\": container with ID starting with 4158ddb88378c0c6a0a213c7a6a66bfacbc34560966f12e812d14ef4d83e927f not found: ID does not exist" containerID="4158ddb88378c0c6a0a213c7a6a66bfacbc34560966f12e812d14ef4d83e927f"
Sep 08 20:55:17.865894 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.865855    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4158ddb88378c0c6a0a213c7a6a66bfacbc34560966f12e812d14ef4d83e927f"} err="failed to get container status \"4158ddb88378c0c6a0a213c7a6a66bfacbc34560966f12e812d14ef4d83e927f\": rpc error: code = NotFound desc = could not find container \"4158ddb88378c0c6a0a213c7a6a66bfacbc34560966f12e812d14ef4d83e927f\": container with ID starting with 4158ddb88378c0c6a0a213c7a6a66bfacbc34560966f12e812d14ef4d83e927f not found: ID does not exist"
Sep 08 20:55:17.865894 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.865873    2580 scope.go:117] "RemoveContainer" containerID="c45f09717675550df07a3d81dbe1265ef005d0955b8c2604ada5d2e78911ee84"
Sep 08 20:55:17.866140 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:55:17.866122    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c45f09717675550df07a3d81dbe1265ef005d0955b8c2604ada5d2e78911ee84\": container with ID starting with c45f09717675550df07a3d81dbe1265ef005d0955b8c2604ada5d2e78911ee84 not found: ID does not exist" containerID="c45f09717675550df07a3d81dbe1265ef005d0955b8c2604ada5d2e78911ee84"
Sep 08 20:55:17.866182 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.866146    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c45f09717675550df07a3d81dbe1265ef005d0955b8c2604ada5d2e78911ee84"} err="failed to get container status \"c45f09717675550df07a3d81dbe1265ef005d0955b8c2604ada5d2e78911ee84\": rpc error: code = NotFound desc = could not find container \"c45f09717675550df07a3d81dbe1265ef005d0955b8c2604ada5d2e78911ee84\": container with ID starting with c45f09717675550df07a3d81dbe1265ef005d0955b8c2604ada5d2e78911ee84 not found: ID does not exist"
Sep 08 20:55:17.870861 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.870830    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"]
Sep 08 20:55:17.878190 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:17.878164    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-78dd9f-openshift-default-5dfdcdf67b-k5j7h"]
Sep 08 20:55:18.923572 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:18.923532    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe4d6ee4-d36e-4929-b356-15fde3dde9aa" path="/var/lib/kubelet/pods/fe4d6ee4-d36e-4929-b356-15fde3dde9aa/volumes"
Sep 08 20:55:23.195451 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.195412    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"]
Sep 08 20:55:23.195923 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.195684    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn" podUID="6bc743e9-36df-4ade-a4f6-91d275f9a05e" containerName="maas-api" containerID="cri-o://aec40e3399d8f33135227c408381063038efc7e70582ee2f40d4655c03c4fbd0" gracePeriod=30
Sep 08 20:55:23.234568 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.234531    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:23.326160 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.326103    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn" podUID="6bc743e9-36df-4ade-a4f6-91d275f9a05e" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.55:8443/health\": dial tcp 10.133.0.55:8443: connect: connection refused"
Sep 08 20:55:23.326951 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.326921    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:23.427503 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.427471    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:23.449743 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.449672    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:55:23.527370 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.527333    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bc743e9-36df-4ade-a4f6-91d275f9a05e-metrics-certs\") pod \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\" (UID: \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\") "
Sep 08 20:55:23.527576 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.527390    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bc743e9-36df-4ade-a4f6-91d275f9a05e-maas-api-tls\") pod \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\" (UID: \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\") "
Sep 08 20:55:23.527576 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.527481    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27dwf\" (UniqueName: \"kubernetes.io/projected/6bc743e9-36df-4ade-a4f6-91d275f9a05e-kube-api-access-27dwf\") pod \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\" (UID: \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\") "
Sep 08 20:55:23.527714 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.527590    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bc743e9-36df-4ade-a4f6-91d275f9a05e-openshift-service-ca\") pod \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\" (UID: \"6bc743e9-36df-4ade-a4f6-91d275f9a05e\") "
Sep 08 20:55:23.527968 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.527934    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bc743e9-36df-4ade-a4f6-91d275f9a05e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6bc743e9-36df-4ade-a4f6-91d275f9a05e" (UID: "6bc743e9-36df-4ade-a4f6-91d275f9a05e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:55:23.529555 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.529529    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bc743e9-36df-4ade-a4f6-91d275f9a05e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6bc743e9-36df-4ade-a4f6-91d275f9a05e" (UID: "6bc743e9-36df-4ade-a4f6-91d275f9a05e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:55:23.529658 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.529552    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bc743e9-36df-4ade-a4f6-91d275f9a05e-kube-api-access-27dwf" (OuterVolumeSpecName: "kube-api-access-27dwf") pod "6bc743e9-36df-4ade-a4f6-91d275f9a05e" (UID: "6bc743e9-36df-4ade-a4f6-91d275f9a05e"). InnerVolumeSpecName "kube-api-access-27dwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:55:23.529658 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.529590    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bc743e9-36df-4ade-a4f6-91d275f9a05e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6bc743e9-36df-4ade-a4f6-91d275f9a05e" (UID: "6bc743e9-36df-4ade-a4f6-91d275f9a05e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 20:55:23.544255 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.544227    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:23.629392 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.629352    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27dwf\" (UniqueName: \"kubernetes.io/projected/6bc743e9-36df-4ade-a4f6-91d275f9a05e-kube-api-access-27dwf\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:23.629392 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.629386    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bc743e9-36df-4ade-a4f6-91d275f9a05e-openshift-service-ca\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:23.629392 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.629397    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6bc743e9-36df-4ade-a4f6-91d275f9a05e-metrics-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:23.629649 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.629405    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bc743e9-36df-4ade-a4f6-91d275f9a05e-maas-api-tls\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:23.879450 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.879420    2580 generic.go:356] "Generic (PLEG): container finished" podID="6bc743e9-36df-4ade-a4f6-91d275f9a05e" containerID="aec40e3399d8f33135227c408381063038efc7e70582ee2f40d4655c03c4fbd0" exitCode=0
Sep 08 20:55:23.879629 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.879467    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"
Sep 08 20:55:23.879629 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.879510    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn" event={"ID":"6bc743e9-36df-4ade-a4f6-91d275f9a05e","Type":"ContainerDied","Data":"aec40e3399d8f33135227c408381063038efc7e70582ee2f40d4655c03c4fbd0"}
Sep 08 20:55:23.879629 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.879566    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn" event={"ID":"6bc743e9-36df-4ade-a4f6-91d275f9a05e","Type":"ContainerDied","Data":"a4b21c49e195b828e6b96038a122975348cdb0e70cb29508cc30b84e81f48694"}
Sep 08 20:55:23.879629 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.879589    2580 scope.go:117] "RemoveContainer" containerID="aec40e3399d8f33135227c408381063038efc7e70582ee2f40d4655c03c4fbd0"
Sep 08 20:55:23.889315 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.889297    2580 scope.go:117] "RemoveContainer" containerID="aec40e3399d8f33135227c408381063038efc7e70582ee2f40d4655c03c4fbd0"
Sep 08 20:55:23.889617 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:55:23.889592    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aec40e3399d8f33135227c408381063038efc7e70582ee2f40d4655c03c4fbd0\": container with ID starting with aec40e3399d8f33135227c408381063038efc7e70582ee2f40d4655c03c4fbd0 not found: ID does not exist" containerID="aec40e3399d8f33135227c408381063038efc7e70582ee2f40d4655c03c4fbd0"
Sep 08 20:55:23.889715 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.889621    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aec40e3399d8f33135227c408381063038efc7e70582ee2f40d4655c03c4fbd0"} err="failed to get container status \"aec40e3399d8f33135227c408381063038efc7e70582ee2f40d4655c03c4fbd0\": rpc error: code = NotFound desc = could not find container \"aec40e3399d8f33135227c408381063038efc7e70582ee2f40d4655c03c4fbd0\": container with ID starting with aec40e3399d8f33135227c408381063038efc7e70582ee2f40d4655c03c4fbd0 not found: ID does not exist"
Sep 08 20:55:23.906129 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.906092    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"]
Sep 08 20:55:23.907969 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.907935    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-c9590b-7f7bd48b44-nzgvn"]
Sep 08 20:55:23.909856 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:23.909832    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:24.160730 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:24.160639    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:24.923304 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:24.923270    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bc743e9-36df-4ade-a4f6-91d275f9a05e" path="/var/lib/kubelet/pods/6bc743e9-36df-4ade-a4f6-91d275f9a05e/volumes"
Sep 08 20:55:29.859177 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:29.859137    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:29.960183 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:29.960140    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:30.144505 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:30.144455    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"]
Sep 08 20:55:30.144729 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:30.144683    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv" podUID="0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" containerName="istio-proxy" containerID="cri-o://a8cbabd9d7b464f9f11278fc34807b69d0a24dff4b45b0911adbc24e9c32a733" gracePeriod=30
Sep 08 20:55:30.163643 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:30.163593    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:55:35.408301 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.408277    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:55:35.541760 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.541653    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-credential-socket\") pod \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") "
Sep 08 20:55:35.541760 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.541715    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istiod-ca-cert\") pod \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") "
Sep 08 20:55:35.541760 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.541731    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-workload-socket\") pod \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") "
Sep 08 20:55:35.541760 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.541753    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-data\") pod \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") "
Sep 08 20:55:35.542155 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.541934    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-token\") pod \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") "
Sep 08 20:55:35.542155 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.541994    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-data" (OuterVolumeSpecName: "istio-data") pod "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" (UID: "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:55:35.542155 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.542013    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" (UID: "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:55:35.542155 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.542069    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-workload-certs\") pod \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") "
Sep 08 20:55:35.542155 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.542096    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqxfm\" (UniqueName: \"kubernetes.io/projected/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-kube-api-access-jqxfm\") pod \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") "
Sep 08 20:55:35.542155 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.542103    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" (UID: "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:55:35.542155 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.542114    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-envoy\") pod \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") "
Sep 08 20:55:35.542155 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.542145    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" (UID: "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 20:55:35.542562 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.542216    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-podinfo\") pod \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\" (UID: \"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef\") "
Sep 08 20:55:35.542562 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.542313    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" (UID: "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:55:35.542562 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.542535    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-workload-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:35.542562 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.542550    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-credential-socket\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:35.542562 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.542560    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istiod-ca-cert\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:35.542819 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.542569    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-workload-socket\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:35.542819 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.542578    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-data\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:35.544311 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.544281    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-token" (OuterVolumeSpecName: "istio-token") pod "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" (UID: "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:55:35.544436 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.544397    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" (UID: "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 20:55:35.544436 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.544405    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" (UID: "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 20:55:35.544556 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.544502    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-kube-api-access-jqxfm" (OuterVolumeSpecName: "kube-api-access-jqxfm") pod "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" (UID: "0c4e36f7-8249-4ccf-8f70-3d608af0b5ef"). InnerVolumeSpecName "kube-api-access-jqxfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 20:55:35.643166 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.643133    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-token\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:35.643166 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.643164    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqxfm\" (UniqueName: \"kubernetes.io/projected/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-kube-api-access-jqxfm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:35.643166 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.643173    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-envoy\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:35.643420 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.643181    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef-istio-podinfo\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 20:55:35.942993 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.942942    2580 generic.go:356] "Generic (PLEG): container finished" podID="0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" containerID="a8cbabd9d7b464f9f11278fc34807b69d0a24dff4b45b0911adbc24e9c32a733" exitCode=0
Sep 08 20:55:35.943181 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.942990    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv" event={"ID":"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef","Type":"ContainerDied","Data":"a8cbabd9d7b464f9f11278fc34807b69d0a24dff4b45b0911adbc24e9c32a733"}
Sep 08 20:55:35.943181 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.943006    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"
Sep 08 20:55:35.943181 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.943045    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv" event={"ID":"0c4e36f7-8249-4ccf-8f70-3d608af0b5ef","Type":"ContainerDied","Data":"2fa83f15241851fad51332e9331fa4a0f36002b52c16c5908ca134640fd48e6b"}
Sep 08 20:55:35.943181 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.943062    2580 scope.go:117] "RemoveContainer" containerID="a8cbabd9d7b464f9f11278fc34807b69d0a24dff4b45b0911adbc24e9c32a733"
Sep 08 20:55:35.953099 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.953081    2580 scope.go:117] "RemoveContainer" containerID="096f5d0e47453fab5034b774df951f15fefb0456fd79c002d0d124717407fd17"
Sep 08 20:55:35.961379 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.961361    2580 scope.go:117] "RemoveContainer" containerID="a8cbabd9d7b464f9f11278fc34807b69d0a24dff4b45b0911adbc24e9c32a733"
Sep 08 20:55:35.961654 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:55:35.961634    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8cbabd9d7b464f9f11278fc34807b69d0a24dff4b45b0911adbc24e9c32a733\": container with ID starting with a8cbabd9d7b464f9f11278fc34807b69d0a24dff4b45b0911adbc24e9c32a733 not found: ID does not exist" containerID="a8cbabd9d7b464f9f11278fc34807b69d0a24dff4b45b0911adbc24e9c32a733"
Sep 08 20:55:35.961704 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.961662    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8cbabd9d7b464f9f11278fc34807b69d0a24dff4b45b0911adbc24e9c32a733"} err="failed to get container status \"a8cbabd9d7b464f9f11278fc34807b69d0a24dff4b45b0911adbc24e9c32a733\": rpc error: code = NotFound desc = could not find container \"a8cbabd9d7b464f9f11278fc34807b69d0a24dff4b45b0911adbc24e9c32a733\": container with ID starting with a8cbabd9d7b464f9f11278fc34807b69d0a24dff4b45b0911adbc24e9c32a733 not found: ID does not exist"
Sep 08 20:55:35.961704 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.961680    2580 scope.go:117] "RemoveContainer" containerID="096f5d0e47453fab5034b774df951f15fefb0456fd79c002d0d124717407fd17"
Sep 08 20:55:35.961971 ip-10-0-136-142 kubenswrapper[2580]: E0908 20:55:35.961940    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"096f5d0e47453fab5034b774df951f15fefb0456fd79c002d0d124717407fd17\": container with ID starting with 096f5d0e47453fab5034b774df951f15fefb0456fd79c002d0d124717407fd17 not found: ID does not exist" containerID="096f5d0e47453fab5034b774df951f15fefb0456fd79c002d0d124717407fd17"
Sep 08 20:55:35.962036 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.961977    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"096f5d0e47453fab5034b774df951f15fefb0456fd79c002d0d124717407fd17"} err="failed to get container status \"096f5d0e47453fab5034b774df951f15fefb0456fd79c002d0d124717407fd17\": rpc error: code = NotFound desc = could not find container \"096f5d0e47453fab5034b774df951f15fefb0456fd79c002d0d124717407fd17\": container with ID starting with 096f5d0e47453fab5034b774df951f15fefb0456fd79c002d0d124717407fd17 not found: ID does not exist"
Sep 08 20:55:35.965130 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.965108    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"]
Sep 08 20:55:35.970239 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:35.970216    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-c9590b-openshift-default-6cb7fccc54-ckntv"]
Sep 08 20:55:36.924167 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:55:36.924134    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" path="/var/lib/kubelet/pods/0c4e36f7-8249-4ccf-8f70-3d608af0b5ef/volumes"
Sep 08 20:56:59.634934 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:56:59.634897    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:57:01.031272 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:57:01.031235    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:59:55.515710 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.515671    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"]
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516036    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe4d6ee4-d36e-4929-b356-15fde3dde9aa" containerName="istio-proxy"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516047    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe4d6ee4-d36e-4929-b356-15fde3dde9aa" containerName="istio-proxy"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516055    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" containerName="istio-proxy"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516063    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" containerName="istio-proxy"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516070    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" containerName="istio-proxy"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516076    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" containerName="istio-proxy"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516082    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="124d18f2-8058-4281-8d27-748adbd924ab" containerName="maas-api"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516088    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="124d18f2-8058-4281-8d27-748adbd924ab" containerName="maas-api"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516095    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68da7896-f3c6-4826-b385-d5c90591a37e" containerName="maas-api"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516100    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="68da7896-f3c6-4826-b385-d5c90591a37e" containerName="maas-api"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516112    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe4d6ee4-d36e-4929-b356-15fde3dde9aa" containerName="istio-proxy"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516119    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe4d6ee4-d36e-4929-b356-15fde3dde9aa" containerName="istio-proxy"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516130    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bc743e9-36df-4ade-a4f6-91d275f9a05e" containerName="maas-api"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516135    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bc743e9-36df-4ade-a4f6-91d275f9a05e" containerName="maas-api"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516140    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e36f9914-fd54-42ce-a2a0-9b9322afac30" containerName="main"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516145    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36f9914-fd54-42ce-a2a0-9b9322afac30" containerName="main"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516202    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" containerName="istio-proxy"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516207    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c4e36f7-8249-4ccf-8f70-3d608af0b5ef" containerName="istio-proxy"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516214    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe4d6ee4-d36e-4929-b356-15fde3dde9aa" containerName="istio-proxy"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516221    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e36f9914-fd54-42ce-a2a0-9b9322afac30" containerName="main"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516226    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="124d18f2-8058-4281-8d27-748adbd924ab" containerName="maas-api"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516232    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="68da7896-f3c6-4826-b385-d5c90591a37e" containerName="maas-api"
Sep 08 20:59:55.516225 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516237    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe4d6ee4-d36e-4929-b356-15fde3dde9aa" containerName="istio-proxy"
Sep 08 20:59:55.516920 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.516243    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bc743e9-36df-4ade-a4f6-91d275f9a05e" containerName="maas-api"
Sep 08 20:59:55.519208 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.519183    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.521786 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.521762    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-89808597-kserve-self-signed-certs\""
Sep 08 20:59:55.530408 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.530380    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"]
Sep 08 20:59:55.640370 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.640338    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-home\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.640570 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.640378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-tmp-dir\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.640570 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.640414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-dshm\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.640570 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.640445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-tls-certs\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.640570 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.640464    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsfhm\" (UniqueName: \"kubernetes.io/projected/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-kube-api-access-tsfhm\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.640749 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.640563    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-model-cache\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.672479 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.672434    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 20:59:55.741659 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.741619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-tls-certs\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.741659 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.741664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsfhm\" (UniqueName: \"kubernetes.io/projected/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-kube-api-access-tsfhm\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.741945 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.741783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-model-cache\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.741945 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.741847    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-home\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.741945 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.741873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-tmp-dir\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.741945 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.741928    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-dshm\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.742266 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.742243    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-model-cache\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.742316 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.742295    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-home\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.742365 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.742308    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-tmp-dir\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.744106 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.744085    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-dshm\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.744301 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.744285    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-tls-certs\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.759523 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.759495    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsfhm\" (UniqueName: \"kubernetes.io/projected/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-kube-api-access-tsfhm\") pod \"e2e-same-id-a-89808597-kserve-56b9968667-xt2sp\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") " pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:55.828208 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:55.828122    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 20:59:56.104712 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:56.104616    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"]
Sep 08 20:59:56.107597 ip-10-0-136-142 kubenswrapper[2580]: W0908 20:59:56.107565    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfad78ee1_ad01_4f77_9ac9_c96ad0b0f3e2.slice/crio-1ae75df4da3a43c50a09860641a462cff7041a236f46f4cc20c36a1f282e0993 WatchSource:0}: Error finding container 1ae75df4da3a43c50a09860641a462cff7041a236f46f4cc20c36a1f282e0993: Status 404 returned error can't find the container with id 1ae75df4da3a43c50a09860641a462cff7041a236f46f4cc20c36a1f282e0993
Sep 08 20:59:56.178019 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:56.177977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp" event={"ID":"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2","Type":"ContainerStarted","Data":"8bcb8fec8576f862139eb5ea01f3ced92deaf83ec651faf1e202b81ba7f09741"}
Sep 08 20:59:56.178188 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:56.178035    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp" event={"ID":"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2","Type":"ContainerStarted","Data":"1ae75df4da3a43c50a09860641a462cff7041a236f46f4cc20c36a1f282e0993"}
Sep 08 20:59:56.199103 ip-10-0-136-142 kubenswrapper[2580]: I0908 20:59:56.199050    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp" podStartSLOduration=1.19903153 podStartE2EDuration="1.19903153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 20:59:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 20:59:56.198024339 +0000 UTC m=+2069.745273830" watchObservedRunningTime="2026-09-08 20:59:56.19903153 +0000 UTC m=+2069.746281011"
Sep 08 21:00:05.828577 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:05.828532    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 21:00:05.829002 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:05.828600    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 21:00:05.841737 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:05.841706    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 21:00:06.237976 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:06.237936    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 21:00:13.133177 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:13.132787    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 21:00:13.233218 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:13.233180    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 21:00:14.811421 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:14.811387    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"]
Sep 08 21:00:14.814797 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:14.814778    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:14.820131 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:14.820105    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-89808597-kserve-self-signed-certs\""
Sep 08 21:00:14.837126 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:14.837093    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"]
Sep 08 21:00:14.884248 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:14.884217    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 21:00:14.916939 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:14.916902    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-home\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:14.916939 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:14.916945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-model-cache\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:14.917185 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:14.917037    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3982e869-7955-4b08-8bd0-21c17997b9b7-tls-certs\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:14.917185 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:14.917131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-dshm\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:14.917185 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:14.917160    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c986\" (UniqueName: \"kubernetes.io/projected/3982e869-7955-4b08-8bd0-21c17997b9b7-kube-api-access-5c986\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:14.917185 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:14.917184    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-tmp-dir\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:15.018334 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:15.018293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c986\" (UniqueName: \"kubernetes.io/projected/3982e869-7955-4b08-8bd0-21c17997b9b7-kube-api-access-5c986\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:15.018533 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:15.018342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-tmp-dir\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:15.018533 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:15.018376    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-home\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:15.018533 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:15.018395    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-model-cache\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:15.018533 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:15.018420    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3982e869-7955-4b08-8bd0-21c17997b9b7-tls-certs\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:15.018533 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:15.018494    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-dshm\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:15.018820 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:15.018787    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-tmp-dir\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:15.018820 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:15.018800    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-model-cache\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:15.018918 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:15.018855    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-home\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:15.020854 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:15.020827    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-dshm\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:15.021128 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:15.021110    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3982e869-7955-4b08-8bd0-21c17997b9b7-tls-certs\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:15.026999 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:15.026950    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c986\" (UniqueName: \"kubernetes.io/projected/3982e869-7955-4b08-8bd0-21c17997b9b7-kube-api-access-5c986\") pod \"e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") " pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:15.124589 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:15.124487    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:15.402735 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:15.402709    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"]
Sep 08 21:00:15.404093 ip-10-0-136-142 kubenswrapper[2580]: W0908 21:00:15.404066    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3982e869_7955_4b08_8bd0_21c17997b9b7.slice/crio-17964cef559114e5c2c47cd86758344fb4ea30d0f09b6d8639bc333b97a454e0 WatchSource:0}: Error finding container 17964cef559114e5c2c47cd86758344fb4ea30d0f09b6d8639bc333b97a454e0: Status 404 returned error can't find the container with id 17964cef559114e5c2c47cd86758344fb4ea30d0f09b6d8639bc333b97a454e0
Sep 08 21:00:16.276470 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:16.276427    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn" event={"ID":"3982e869-7955-4b08-8bd0-21c17997b9b7","Type":"ContainerStarted","Data":"d7a5b7d0f1aa444d3d847bb214af4a8b98a5790620b698246c09063218be2f06"}
Sep 08 21:00:16.276470 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:16.276475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn" event={"ID":"3982e869-7955-4b08-8bd0-21c17997b9b7","Type":"ContainerStarted","Data":"17964cef559114e5c2c47cd86758344fb4ea30d0f09b6d8639bc333b97a454e0"}
Sep 08 21:00:25.124852 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:25.124816    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:25.125329 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:25.124866    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:25.137931 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:25.137899    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:25.168153 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:25.168096    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn" podStartSLOduration=11.16807708 podStartE2EDuration="11.16807708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 21:00:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 21:00:16.308891108 +0000 UTC m=+2089.856140588" watchObservedRunningTime="2026-09-08 21:00:25.16807708 +0000 UTC m=+2098.715326615"
Sep 08 21:00:25.331211 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:25.331180    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:31.934811 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:31.934774    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"]
Sep 08 21:00:31.963556 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:31.963509    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 21:00:33.195802 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:33.195764    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"]
Sep 08 21:00:33.210143 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:33.210089    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 21:00:45.044128 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:45.044079    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 21:00:45.146896 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:45.146858    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 21:00:46.954381 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:46.954334    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp" podUID="fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2" containerName="main" containerID="cri-o://8bcb8fec8576f862139eb5ea01f3ced92deaf83ec651faf1e202b81ba7f09741" gracePeriod=45
Sep 08 21:00:47.202497 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.202472    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 21:00:47.226846 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.226761    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-home\") pod \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") "
Sep 08 21:00:47.226846 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.226834    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-dshm\") pod \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") "
Sep 08 21:00:47.227080 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.226853    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-model-cache\") pod \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") "
Sep 08 21:00:47.227080 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.226884    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsfhm\" (UniqueName: \"kubernetes.io/projected/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-kube-api-access-tsfhm\") pod \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") "
Sep 08 21:00:47.227080 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.226915    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-tmp-dir\") pod \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") "
Sep 08 21:00:47.227080 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.226944    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-tls-certs\") pod \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\" (UID: \"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2\") "
Sep 08 21:00:47.227288 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.227092    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-home" (OuterVolumeSpecName: "home") pod "fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2" (UID: "fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 21:00:47.227288 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.227237    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-home\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 21:00:47.227288 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.227234    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-model-cache" (OuterVolumeSpecName: "model-cache") pod "fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2" (UID: "fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 21:00:47.227402 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.227279    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2" (UID: "fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 21:00:47.229123 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.229089    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2" (UID: "fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 21:00:47.229123 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.229099    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-kube-api-access-tsfhm" (OuterVolumeSpecName: "kube-api-access-tsfhm") pod "fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2" (UID: "fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2"). InnerVolumeSpecName "kube-api-access-tsfhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 21:00:47.229261 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.229147    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-dshm" (OuterVolumeSpecName: "dshm") pod "fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2" (UID: "fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 21:00:47.328033 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.327999    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-dshm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 21:00:47.328033 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.328030    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-model-cache\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 21:00:47.328033 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.328041    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsfhm\" (UniqueName: \"kubernetes.io/projected/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-kube-api-access-tsfhm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 21:00:47.328280 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.328052    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-tmp-dir\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 21:00:47.328280 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.328061    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2-tls-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 21:00:47.420966 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.420927    2580 generic.go:356] "Generic (PLEG): container finished" podID="fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2" containerID="8bcb8fec8576f862139eb5ea01f3ced92deaf83ec651faf1e202b81ba7f09741" exitCode=0
Sep 08 21:00:47.421143 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.421003    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"
Sep 08 21:00:47.421143 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.421001    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp" event={"ID":"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2","Type":"ContainerDied","Data":"8bcb8fec8576f862139eb5ea01f3ced92deaf83ec651faf1e202b81ba7f09741"}
Sep 08 21:00:47.421143 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.421047    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp" event={"ID":"fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2","Type":"ContainerDied","Data":"1ae75df4da3a43c50a09860641a462cff7041a236f46f4cc20c36a1f282e0993"}
Sep 08 21:00:47.421143 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.421069    2580 scope.go:117] "RemoveContainer" containerID="8bcb8fec8576f862139eb5ea01f3ced92deaf83ec651faf1e202b81ba7f09741"
Sep 08 21:00:47.433525 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.433504    2580 scope.go:117] "RemoveContainer" containerID="8bcb8fec8576f862139eb5ea01f3ced92deaf83ec651faf1e202b81ba7f09741"
Sep 08 21:00:47.433877 ip-10-0-136-142 kubenswrapper[2580]: E0908 21:00:47.433856    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bcb8fec8576f862139eb5ea01f3ced92deaf83ec651faf1e202b81ba7f09741\": container with ID starting with 8bcb8fec8576f862139eb5ea01f3ced92deaf83ec651faf1e202b81ba7f09741 not found: ID does not exist" containerID="8bcb8fec8576f862139eb5ea01f3ced92deaf83ec651faf1e202b81ba7f09741"
Sep 08 21:00:47.433952 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.433885    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bcb8fec8576f862139eb5ea01f3ced92deaf83ec651faf1e202b81ba7f09741"} err="failed to get container status \"8bcb8fec8576f862139eb5ea01f3ced92deaf83ec651faf1e202b81ba7f09741\": rpc error: code = NotFound desc = could not find container \"8bcb8fec8576f862139eb5ea01f3ced92deaf83ec651faf1e202b81ba7f09741\": container with ID starting with 8bcb8fec8576f862139eb5ea01f3ced92deaf83ec651faf1e202b81ba7f09741 not found: ID does not exist"
Sep 08 21:00:47.455031 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.455000    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"]
Sep 08 21:00:47.463627 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:47.463587    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-89808597-kserve-56b9968667-xt2sp"]
Sep 08 21:00:48.216461 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.216418    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn" podUID="3982e869-7955-4b08-8bd0-21c17997b9b7" containerName="main" containerID="cri-o://d7a5b7d0f1aa444d3d847bb214af4a8b98a5790620b698246c09063218be2f06" gracePeriod=45
Sep 08 21:00:48.427676 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.427643    2580 generic.go:356] "Generic (PLEG): container finished" podID="3982e869-7955-4b08-8bd0-21c17997b9b7" containerID="d7a5b7d0f1aa444d3d847bb214af4a8b98a5790620b698246c09063218be2f06" exitCode=0
Sep 08 21:00:48.427852 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.427715    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn" event={"ID":"3982e869-7955-4b08-8bd0-21c17997b9b7","Type":"ContainerDied","Data":"d7a5b7d0f1aa444d3d847bb214af4a8b98a5790620b698246c09063218be2f06"}
Sep 08 21:00:48.467658 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.467581    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:48.538711 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.538680    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3982e869-7955-4b08-8bd0-21c17997b9b7-tls-certs\") pod \"3982e869-7955-4b08-8bd0-21c17997b9b7\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") "
Sep 08 21:00:48.538711 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.538716    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-model-cache\") pod \"3982e869-7955-4b08-8bd0-21c17997b9b7\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") "
Sep 08 21:00:48.538982 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.538740    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5c986\" (UniqueName: \"kubernetes.io/projected/3982e869-7955-4b08-8bd0-21c17997b9b7-kube-api-access-5c986\") pod \"3982e869-7955-4b08-8bd0-21c17997b9b7\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") "
Sep 08 21:00:48.538982 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.538755    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-home\") pod \"3982e869-7955-4b08-8bd0-21c17997b9b7\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") "
Sep 08 21:00:48.538982 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.538778    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-dshm\") pod \"3982e869-7955-4b08-8bd0-21c17997b9b7\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") "
Sep 08 21:00:48.538982 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.538829    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-tmp-dir\") pod \"3982e869-7955-4b08-8bd0-21c17997b9b7\" (UID: \"3982e869-7955-4b08-8bd0-21c17997b9b7\") "
Sep 08 21:00:48.539197 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.539062    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-home" (OuterVolumeSpecName: "home") pod "3982e869-7955-4b08-8bd0-21c17997b9b7" (UID: "3982e869-7955-4b08-8bd0-21c17997b9b7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 21:00:48.539197 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.539078    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-model-cache" (OuterVolumeSpecName: "model-cache") pod "3982e869-7955-4b08-8bd0-21c17997b9b7" (UID: "3982e869-7955-4b08-8bd0-21c17997b9b7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 21:00:48.539197 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.539192    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3982e869-7955-4b08-8bd0-21c17997b9b7" (UID: "3982e869-7955-4b08-8bd0-21c17997b9b7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 21:00:48.540942 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.540915    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3982e869-7955-4b08-8bd0-21c17997b9b7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3982e869-7955-4b08-8bd0-21c17997b9b7" (UID: "3982e869-7955-4b08-8bd0-21c17997b9b7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 21:00:48.540942 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.540919    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-dshm" (OuterVolumeSpecName: "dshm") pod "3982e869-7955-4b08-8bd0-21c17997b9b7" (UID: "3982e869-7955-4b08-8bd0-21c17997b9b7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 21:00:48.541098 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.541002    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3982e869-7955-4b08-8bd0-21c17997b9b7-kube-api-access-5c986" (OuterVolumeSpecName: "kube-api-access-5c986") pod "3982e869-7955-4b08-8bd0-21c17997b9b7" (UID: "3982e869-7955-4b08-8bd0-21c17997b9b7"). InnerVolumeSpecName "kube-api-access-5c986". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 21:00:48.639839 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.639795    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-tmp-dir\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 21:00:48.639839 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.639831    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3982e869-7955-4b08-8bd0-21c17997b9b7-tls-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 21:00:48.639839 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.639840    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-model-cache\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 21:00:48.639839 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.639849    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5c986\" (UniqueName: \"kubernetes.io/projected/3982e869-7955-4b08-8bd0-21c17997b9b7-kube-api-access-5c986\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 21:00:48.640148 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.639857    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-home\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 21:00:48.640148 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.639865    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3982e869-7955-4b08-8bd0-21c17997b9b7-dshm\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 21:00:48.927147 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:48.925603    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2" path="/var/lib/kubelet/pods/fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2/volumes"
Sep 08 21:00:49.435350 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:49.435313    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn" event={"ID":"3982e869-7955-4b08-8bd0-21c17997b9b7","Type":"ContainerDied","Data":"17964cef559114e5c2c47cd86758344fb4ea30d0f09b6d8639bc333b97a454e0"}
Sep 08 21:00:49.435350 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:49.435341    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"
Sep 08 21:00:49.435903 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:49.435372    2580 scope.go:117] "RemoveContainer" containerID="d7a5b7d0f1aa444d3d847bb214af4a8b98a5790620b698246c09063218be2f06"
Sep 08 21:00:49.461263 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:49.461232    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"]
Sep 08 21:00:49.467911 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:49.467884    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-89808597-kserve-5788bcf46c-rc7gn"]
Sep 08 21:00:50.918027 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:50.917985    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"]
Sep 08 21:00:50.918495 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:50.918379    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2" containerName="main"
Sep 08 21:00:50.918495 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:50.918393    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2" containerName="main"
Sep 08 21:00:50.918495 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:50.918401    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3982e869-7955-4b08-8bd0-21c17997b9b7" containerName="main"
Sep 08 21:00:50.918495 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:50.918411    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3982e869-7955-4b08-8bd0-21c17997b9b7" containerName="main"
Sep 08 21:00:50.918495 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:50.918489    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3982e869-7955-4b08-8bd0-21c17997b9b7" containerName="main"
Sep 08 21:00:50.918706 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:50.918501    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fad78ee1-ad01-4f77-9ac9-c96ad0b0f3e2" containerName="main"
Sep 08 21:00:50.922893 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:50.922866    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"
Sep 08 21:00:50.929042 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:50.929014    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3982e869-7955-4b08-8bd0-21c17997b9b7" path="/var/lib/kubelet/pods/3982e869-7955-4b08-8bd0-21c17997b9b7/volumes"
Sep 08 21:00:50.962173 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:50.962140    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"]
Sep 08 21:00:51.057558 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:51.057512    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98428659-81f6-4bdd-bf67-7cc063a6622b-maas-api-tls\") pod \"maas-api-6b8bc4d5d4-d5h2s\" (UID: \"98428659-81f6-4bdd-bf67-7cc063a6622b\") " pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"
Sep 08 21:00:51.057778 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:51.057650    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srtg8\" (UniqueName: \"kubernetes.io/projected/98428659-81f6-4bdd-bf67-7cc063a6622b-kube-api-access-srtg8\") pod \"maas-api-6b8bc4d5d4-d5h2s\" (UID: \"98428659-81f6-4bdd-bf67-7cc063a6622b\") " pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"
Sep 08 21:00:51.057778 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:51.057759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98428659-81f6-4bdd-bf67-7cc063a6622b-openshift-service-ca\") pod \"maas-api-6b8bc4d5d4-d5h2s\" (UID: \"98428659-81f6-4bdd-bf67-7cc063a6622b\") " pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"
Sep 08 21:00:51.057892 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:51.057788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98428659-81f6-4bdd-bf67-7cc063a6622b-metrics-certs\") pod \"maas-api-6b8bc4d5d4-d5h2s\" (UID: \"98428659-81f6-4bdd-bf67-7cc063a6622b\") " pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"
Sep 08 21:00:51.158813 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:51.158772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98428659-81f6-4bdd-bf67-7cc063a6622b-openshift-service-ca\") pod \"maas-api-6b8bc4d5d4-d5h2s\" (UID: \"98428659-81f6-4bdd-bf67-7cc063a6622b\") " pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"
Sep 08 21:00:51.158813 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:51.158813    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98428659-81f6-4bdd-bf67-7cc063a6622b-metrics-certs\") pod \"maas-api-6b8bc4d5d4-d5h2s\" (UID: \"98428659-81f6-4bdd-bf67-7cc063a6622b\") " pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"
Sep 08 21:00:51.159052 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:51.158850    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98428659-81f6-4bdd-bf67-7cc063a6622b-maas-api-tls\") pod \"maas-api-6b8bc4d5d4-d5h2s\" (UID: \"98428659-81f6-4bdd-bf67-7cc063a6622b\") " pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"
Sep 08 21:00:51.159052 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:51.158909    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srtg8\" (UniqueName: \"kubernetes.io/projected/98428659-81f6-4bdd-bf67-7cc063a6622b-kube-api-access-srtg8\") pod \"maas-api-6b8bc4d5d4-d5h2s\" (UID: \"98428659-81f6-4bdd-bf67-7cc063a6622b\") " pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"
Sep 08 21:00:51.159471 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:51.159441    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98428659-81f6-4bdd-bf67-7cc063a6622b-openshift-service-ca\") pod \"maas-api-6b8bc4d5d4-d5h2s\" (UID: \"98428659-81f6-4bdd-bf67-7cc063a6622b\") " pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"
Sep 08 21:00:51.161409 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:51.161389    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98428659-81f6-4bdd-bf67-7cc063a6622b-maas-api-tls\") pod \"maas-api-6b8bc4d5d4-d5h2s\" (UID: \"98428659-81f6-4bdd-bf67-7cc063a6622b\") " pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"
Sep 08 21:00:51.161508 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:51.161427    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98428659-81f6-4bdd-bf67-7cc063a6622b-metrics-certs\") pod \"maas-api-6b8bc4d5d4-d5h2s\" (UID: \"98428659-81f6-4bdd-bf67-7cc063a6622b\") " pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"
Sep 08 21:00:51.168971 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:51.168896    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srtg8\" (UniqueName: \"kubernetes.io/projected/98428659-81f6-4bdd-bf67-7cc063a6622b-kube-api-access-srtg8\") pod \"maas-api-6b8bc4d5d4-d5h2s\" (UID: \"98428659-81f6-4bdd-bf67-7cc063a6622b\") " pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"
Sep 08 21:00:51.233372 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:51.233333    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"
Sep 08 21:00:51.511287 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:51.511259    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"]
Sep 08 21:00:51.513998 ip-10-0-136-142 kubenswrapper[2580]: W0908 21:00:51.513971    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98428659_81f6_4bdd_bf67_7cc063a6622b.slice/crio-67b888fc9b76b09590b110c2194c39ab7d148753999a3f1d063e4e29985cdf63 WatchSource:0}: Error finding container 67b888fc9b76b09590b110c2194c39ab7d148753999a3f1d063e4e29985cdf63: Status 404 returned error can't find the container with id 67b888fc9b76b09590b110c2194c39ab7d148753999a3f1d063e4e29985cdf63
Sep 08 21:00:51.515498 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:51.515474    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 21:00:52.452127 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:52.452092    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s" event={"ID":"98428659-81f6-4bdd-bf67-7cc063a6622b","Type":"ContainerStarted","Data":"807f939f2e391145588b1cc454fe1b65358dbdd2e9da20c0d50603a7987f4166"}
Sep 08 21:00:52.452636 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:52.452140    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s" event={"ID":"98428659-81f6-4bdd-bf67-7cc063a6622b","Type":"ContainerStarted","Data":"67b888fc9b76b09590b110c2194c39ab7d148753999a3f1d063e4e29985cdf63"}
Sep 08 21:00:52.452636 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:52.452216    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"
Sep 08 21:00:52.477452 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:52.477393    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s" podStartSLOduration=2.059903782 podStartE2EDuration="2.47737359s" totalImagesPullingTime="417.469808ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 21:00:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 21:00:52.475819283 +0000 UTC m=+2126.023068765" watchObservedRunningTime="2026-09-08 21:00:52.47737359 +0000 UTC m=+2126.024623071"
Sep 08 21:00:58.462653 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:58.462622    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6b8bc4d5d4-d5h2s"
Sep 08 21:00:58.547372 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:58.547338    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"]
Sep 08 21:00:58.547568 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:58.547544    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn" podUID="121a28e5-e356-4205-8c21-2ac12d541697" containerName="maas-api" containerID="cri-o://6e1d5bd46a76d3e37516b1ab211392960e090a208c9a918ecb2501b110d811c3" gracePeriod=30
Sep 08 21:00:59.480085 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:59.480038    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn" podUID="121a28e5-e356-4205-8c21-2ac12d541697" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.35:8443/health\": dial tcp 10.133.0.35:8443: connect: connection refused"
Sep 08 21:00:59.797132 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:59.797107    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 21:00:59.828209 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:59.828170    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b66hs\" (UniqueName: \"kubernetes.io/projected/121a28e5-e356-4205-8c21-2ac12d541697-kube-api-access-b66hs\") pod \"121a28e5-e356-4205-8c21-2ac12d541697\" (UID: \"121a28e5-e356-4205-8c21-2ac12d541697\") "
Sep 08 21:00:59.828379 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:59.828269    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/121a28e5-e356-4205-8c21-2ac12d541697-maas-api-tls\") pod \"121a28e5-e356-4205-8c21-2ac12d541697\" (UID: \"121a28e5-e356-4205-8c21-2ac12d541697\") "
Sep 08 21:00:59.828379 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:59.828310    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/121a28e5-e356-4205-8c21-2ac12d541697-metrics-certs\") pod \"121a28e5-e356-4205-8c21-2ac12d541697\" (UID: \"121a28e5-e356-4205-8c21-2ac12d541697\") "
Sep 08 21:00:59.828379 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:59.828347    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/121a28e5-e356-4205-8c21-2ac12d541697-openshift-service-ca\") pod \"121a28e5-e356-4205-8c21-2ac12d541697\" (UID: \"121a28e5-e356-4205-8c21-2ac12d541697\") "
Sep 08 21:00:59.828823 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:59.828791    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/121a28e5-e356-4205-8c21-2ac12d541697-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "121a28e5-e356-4205-8c21-2ac12d541697" (UID: "121a28e5-e356-4205-8c21-2ac12d541697"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 21:00:59.830709 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:59.830674    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/121a28e5-e356-4205-8c21-2ac12d541697-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "121a28e5-e356-4205-8c21-2ac12d541697" (UID: "121a28e5-e356-4205-8c21-2ac12d541697"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 21:00:59.830709 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:59.830698    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/121a28e5-e356-4205-8c21-2ac12d541697-kube-api-access-b66hs" (OuterVolumeSpecName: "kube-api-access-b66hs") pod "121a28e5-e356-4205-8c21-2ac12d541697" (UID: "121a28e5-e356-4205-8c21-2ac12d541697"). InnerVolumeSpecName "kube-api-access-b66hs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 21:00:59.830881 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:59.830704    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/121a28e5-e356-4205-8c21-2ac12d541697-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "121a28e5-e356-4205-8c21-2ac12d541697" (UID: "121a28e5-e356-4205-8c21-2ac12d541697"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 21:00:59.929806 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:59.929763    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/121a28e5-e356-4205-8c21-2ac12d541697-maas-api-tls\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 21:00:59.929806 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:59.929805    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/121a28e5-e356-4205-8c21-2ac12d541697-metrics-certs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 21:00:59.929806 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:59.929817    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/121a28e5-e356-4205-8c21-2ac12d541697-openshift-service-ca\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 21:00:59.929806 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:00:59.929826    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b66hs\" (UniqueName: \"kubernetes.io/projected/121a28e5-e356-4205-8c21-2ac12d541697-kube-api-access-b66hs\") on node \"ip-10-0-136-142.ec2.internal\" DevicePath \"\""
Sep 08 21:01:00.491110 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:00.491071    2580 generic.go:356] "Generic (PLEG): container finished" podID="121a28e5-e356-4205-8c21-2ac12d541697" containerID="6e1d5bd46a76d3e37516b1ab211392960e090a208c9a918ecb2501b110d811c3" exitCode=0
Sep 08 21:01:00.491522 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:00.491138    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"
Sep 08 21:01:00.491522 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:00.491157    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn" event={"ID":"121a28e5-e356-4205-8c21-2ac12d541697","Type":"ContainerDied","Data":"6e1d5bd46a76d3e37516b1ab211392960e090a208c9a918ecb2501b110d811c3"}
Sep 08 21:01:00.491522 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:00.491239    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn" event={"ID":"121a28e5-e356-4205-8c21-2ac12d541697","Type":"ContainerDied","Data":"5841d5e8f1c77abc796d21439b8cf7adbf458471a337d1b8785c587a6618639f"}
Sep 08 21:01:00.491522 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:00.491262    2580 scope.go:117] "RemoveContainer" containerID="6e1d5bd46a76d3e37516b1ab211392960e090a208c9a918ecb2501b110d811c3"
Sep 08 21:01:00.500674 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:00.500655    2580 scope.go:117] "RemoveContainer" containerID="6e1d5bd46a76d3e37516b1ab211392960e090a208c9a918ecb2501b110d811c3"
Sep 08 21:01:00.501022 ip-10-0-136-142 kubenswrapper[2580]: E0908 21:01:00.500999    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e1d5bd46a76d3e37516b1ab211392960e090a208c9a918ecb2501b110d811c3\": container with ID starting with 6e1d5bd46a76d3e37516b1ab211392960e090a208c9a918ecb2501b110d811c3 not found: ID does not exist" containerID="6e1d5bd46a76d3e37516b1ab211392960e090a208c9a918ecb2501b110d811c3"
Sep 08 21:01:00.501083 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:00.501031    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e1d5bd46a76d3e37516b1ab211392960e090a208c9a918ecb2501b110d811c3"} err="failed to get container status \"6e1d5bd46a76d3e37516b1ab211392960e090a208c9a918ecb2501b110d811c3\": rpc error: code = NotFound desc = could not find container \"6e1d5bd46a76d3e37516b1ab211392960e090a208c9a918ecb2501b110d811c3\": container with ID starting with 6e1d5bd46a76d3e37516b1ab211392960e090a208c9a918ecb2501b110d811c3 not found: ID does not exist"
Sep 08 21:01:00.512976 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:00.512932    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"]
Sep 08 21:01:00.518007 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:00.517975    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b96cdc7c6-hm2sn"]
Sep 08 21:01:00.924282 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:00.924246    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="121a28e5-e356-4205-8c21-2ac12d541697" path="/var/lib/kubelet/pods/121a28e5-e356-4205-8c21-2ac12d541697/volumes"
Sep 08 21:01:20.943912 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:20.943835    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b8c6c7bc8-mg4qh_e3627210-e1cc-4535-ab72-c888e373bc65/authorino/0.log"
Sep 08 21:01:27.128084 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:27.128052    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-75tzr_e235df69-f4bb-4205-8e69-bccafc3b1eec/server/0.log"
Sep 08 21:01:27.239937 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:27.239908    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-tvxbb_30c86469-f125-4b88-9219-11e5745f58c1/manager/0.log"
Sep 08 21:01:27.624779 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:27.624700    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbd9f8d48-gslsc_2b8d1349-f513-443d-91a4-fa50453cc156/copy-manifests/0.log"
Sep 08 21:01:27.632540 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:27.632513    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbd9f8d48-gslsc_2b8d1349-f513-443d-91a4-fa50453cc156/manager/0.log"
Sep 08 21:01:27.976833 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:27.976803    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6b8bc4d5d4-d5h2s_98428659-81f6-4bdd-bf67-7cc063a6622b/maas-api/0.log"
Sep 08 21:01:29.377870 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:29.377839    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b8c6c7bc8-mg4qh_e3627210-e1cc-4535-ab72-c888e373bc65/authorino/0.log"
Sep 08 21:01:30.557416 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:30.557378    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7_f5be66a0-cfc9-4745-a2a6-83f195bfe307/istio-proxy/0.log"
Sep 08 21:01:30.823798 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:30.823724    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bf757d869-l5cr4_e6468cbd-3da7-4625-80d4-8c4c2ef96d89/kube-auth-proxy/0.log"
Sep 08 21:01:31.992865 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:31.992829    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-nn7g8_59008731-c2ba-4a82-a842-aea570790e12/main/0.log"
Sep 08 21:01:32.601427 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:32.601397    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-t7v26_0f7449da-bd90-4bd6-b14b-3953c3fb0cf1/main/0.log"
Sep 08 21:01:38.339424 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:38.339365    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 21:01:38.526707 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:38.526675    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 21:01:38.613151 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:38.613059    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log" limit=1024
Sep 08 21:01:41.514805 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:41.514776    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fkjc4_a9304897-72a4-4daa-aa6e-9397a440502e/global-pull-secret-syncer/0.log"
Sep 08 21:01:41.611174 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:41.611147    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g26sh_15838251-a64d-4585-831a-69d5bb59336b/konnectivity-agent/0.log"
Sep 08 21:01:41.755165 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:41.755137    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-142.ec2.internal_4dade65b3f03b9eb8dc1b57ffacbb7fd/haproxy/0.log"
Sep 08 21:01:46.216055 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:46.215979    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b8c6c7bc8-mg4qh_e3627210-e1cc-4535-ab72-c888e373bc65/authorino/0.log"
Sep 08 21:01:48.152615 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:48.152585    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-cl6hs_d2ed3c01-5a4d-475b-84f3-97c7969c52da/cluster-monitoring-operator/0.log"
Sep 08 21:01:48.271381 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:48.271349    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7b9ff5f76f-f5nhx_aca1d922-fe5e-462d-a550-e29a90c25ae6/metrics-server/0.log"
Sep 08 21:01:48.329035 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:48.329006    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-wgmpq_d30e5889-153d-4d96-bcd2-9b448b4dbcd1/monitoring-plugin/0.log"
Sep 08 21:01:48.355441 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:48.355412    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6sfwv_ca31dae6-e536-465d-83e6-6a0f18ecadbf/node-exporter/0.log"
Sep 08 21:01:48.383095 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:48.383061    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6sfwv_ca31dae6-e536-465d-83e6-6a0f18ecadbf/kube-rbac-proxy/0.log"
Sep 08 21:01:48.405878 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:48.405804    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6sfwv_ca31dae6-e536-465d-83e6-6a0f18ecadbf/init-textfile/0.log"
Sep 08 21:01:49.157572 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:49.157543    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-qlx76_54f2231d-9338-406a-bb23-e3236edd4f60/prometheus-operator-admission-webhook/0.log"
Sep 08 21:01:49.305295 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:49.305267    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c78bc8cf7-z77f8_4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5/thanos-query/0.log"
Sep 08 21:01:49.331729 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:49.331699    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c78bc8cf7-z77f8_4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5/kube-rbac-proxy-web/0.log"
Sep 08 21:01:49.356991 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:49.356948    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c78bc8cf7-z77f8_4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5/kube-rbac-proxy/0.log"
Sep 08 21:01:49.381544 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:49.381516    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c78bc8cf7-z77f8_4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5/prom-label-proxy/0.log"
Sep 08 21:01:49.405972 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:49.405924    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c78bc8cf7-z77f8_4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5/kube-rbac-proxy-rules/0.log"
Sep 08 21:01:49.430723 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:49.430647    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c78bc8cf7-z77f8_4f763e45-1fe8-42f9-aa7b-6cf79ec7dae5/kube-rbac-proxy-metrics/0.log"
Sep 08 21:01:52.005540 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.005499    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"]
Sep 08 21:01:52.006042 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.006014    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="121a28e5-e356-4205-8c21-2ac12d541697" containerName="maas-api"
Sep 08 21:01:52.006042 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.006032    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="121a28e5-e356-4205-8c21-2ac12d541697" containerName="maas-api"
Sep 08 21:01:52.006171 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.006150    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="121a28e5-e356-4205-8c21-2ac12d541697" containerName="maas-api"
Sep 08 21:01:52.009248 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.009226    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.011424 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.011401    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-w75tg\"/\"default-dockercfg-xmph6\""
Sep 08 21:01:52.011525 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.011401    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w75tg\"/\"openshift-service-ca.crt\""
Sep 08 21:01:52.011974 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.011943    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w75tg\"/\"kube-root-ca.crt\""
Sep 08 21:01:52.017029 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.017006    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"]
Sep 08 21:01:52.088402 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.088365    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6de710ff-43d9-4262-b351-6b28d617e6f1-lib-modules\") pod \"perf-node-gather-daemonset-qdll4\" (UID: \"6de710ff-43d9-4262-b351-6b28d617e6f1\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.088402 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.088409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6de710ff-43d9-4262-b351-6b28d617e6f1-proc\") pod \"perf-node-gather-daemonset-qdll4\" (UID: \"6de710ff-43d9-4262-b351-6b28d617e6f1\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.088638 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.088437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6de710ff-43d9-4262-b351-6b28d617e6f1-sys\") pod \"perf-node-gather-daemonset-qdll4\" (UID: \"6de710ff-43d9-4262-b351-6b28d617e6f1\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.088638 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.088509    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6de710ff-43d9-4262-b351-6b28d617e6f1-podres\") pod \"perf-node-gather-daemonset-qdll4\" (UID: \"6de710ff-43d9-4262-b351-6b28d617e6f1\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.088638 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.088596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxspr\" (UniqueName: \"kubernetes.io/projected/6de710ff-43d9-4262-b351-6b28d617e6f1-kube-api-access-rxspr\") pod \"perf-node-gather-daemonset-qdll4\" (UID: \"6de710ff-43d9-4262-b351-6b28d617e6f1\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.189700 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.189662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6de710ff-43d9-4262-b351-6b28d617e6f1-lib-modules\") pod \"perf-node-gather-daemonset-qdll4\" (UID: \"6de710ff-43d9-4262-b351-6b28d617e6f1\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.189883 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.189710    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6de710ff-43d9-4262-b351-6b28d617e6f1-proc\") pod \"perf-node-gather-daemonset-qdll4\" (UID: \"6de710ff-43d9-4262-b351-6b28d617e6f1\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.189883 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.189740    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6de710ff-43d9-4262-b351-6b28d617e6f1-sys\") pod \"perf-node-gather-daemonset-qdll4\" (UID: \"6de710ff-43d9-4262-b351-6b28d617e6f1\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.189883 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.189762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6de710ff-43d9-4262-b351-6b28d617e6f1-podres\") pod \"perf-node-gather-daemonset-qdll4\" (UID: \"6de710ff-43d9-4262-b351-6b28d617e6f1\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.189883 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.189864    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6de710ff-43d9-4262-b351-6b28d617e6f1-lib-modules\") pod \"perf-node-gather-daemonset-qdll4\" (UID: \"6de710ff-43d9-4262-b351-6b28d617e6f1\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.190054 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.189882    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6de710ff-43d9-4262-b351-6b28d617e6f1-proc\") pod \"perf-node-gather-daemonset-qdll4\" (UID: \"6de710ff-43d9-4262-b351-6b28d617e6f1\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.190054 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.189895    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6de710ff-43d9-4262-b351-6b28d617e6f1-podres\") pod \"perf-node-gather-daemonset-qdll4\" (UID: \"6de710ff-43d9-4262-b351-6b28d617e6f1\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.190054 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.189882    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6de710ff-43d9-4262-b351-6b28d617e6f1-sys\") pod \"perf-node-gather-daemonset-qdll4\" (UID: \"6de710ff-43d9-4262-b351-6b28d617e6f1\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.190054 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.189862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxspr\" (UniqueName: \"kubernetes.io/projected/6de710ff-43d9-4262-b351-6b28d617e6f1-kube-api-access-rxspr\") pod \"perf-node-gather-daemonset-qdll4\" (UID: \"6de710ff-43d9-4262-b351-6b28d617e6f1\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.198007 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.197978    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxspr\" (UniqueName: \"kubernetes.io/projected/6de710ff-43d9-4262-b351-6b28d617e6f1-kube-api-access-rxspr\") pod \"perf-node-gather-daemonset-qdll4\" (UID: \"6de710ff-43d9-4262-b351-6b28d617e6f1\") " pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.320329 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.320239    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.591933 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.591907    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"]
Sep 08 21:01:52.594432 ip-10-0-136-142 kubenswrapper[2580]: W0908 21:01:52.594408    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6de710ff_43d9_4262_b351_6b28d617e6f1.slice/crio-121bb883ad5c47ecc931fae9b9716be7ab33827f93b168d8580fbf7e10d80ca3 WatchSource:0}: Error finding container 121bb883ad5c47ecc931fae9b9716be7ab33827f93b168d8580fbf7e10d80ca3: Status 404 returned error can't find the container with id 121bb883ad5c47ecc931fae9b9716be7ab33827f93b168d8580fbf7e10d80ca3
Sep 08 21:01:52.741205 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.741168    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4" event={"ID":"6de710ff-43d9-4262-b351-6b28d617e6f1","Type":"ContainerStarted","Data":"268234518bee077f05523684e626edc04d5c8ac577bdee2484c705b1a5a4b3bc"}
Sep 08 21:01:52.741396 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.741219    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4" event={"ID":"6de710ff-43d9-4262-b351-6b28d617e6f1","Type":"ContainerStarted","Data":"121bb883ad5c47ecc931fae9b9716be7ab33827f93b168d8580fbf7e10d80ca3"}
Sep 08 21:01:52.741396 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.741302    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:01:52.758316 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.758254    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4" podStartSLOduration=1.758238972 podStartE2EDuration="1.758238972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 21:01:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 21:01:52.75629246 +0000 UTC m=+2186.303541942" watchObservedRunningTime="2026-09-08 21:01:52.758238972 +0000 UTC m=+2186.305488477"
Sep 08 21:01:52.915934 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:52.915889    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-glk4q_5e7ff65d-5192-457e-aac7-8eef1a56d725/volume-data-source-validator/0.log"
Sep 08 21:01:54.077264 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:54.077232    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-75k8c_34b69620-f3c4-471b-85d5-887e22e90910/dns/0.log"
Sep 08 21:01:54.098735 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:54.098710    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-75k8c_34b69620-f3c4-471b-85d5-887e22e90910/kube-rbac-proxy/0.log"
Sep 08 21:01:54.207114 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:54.207084    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cljhb_de02ff1a-312a-4069-b121-dff1aa3b9354/dns-node-resolver/0.log"
Sep 08 21:01:54.784232 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:54.784198    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-79f5ff44f6-sgnmg_52258924-6ad4-4c1b-bb14-1848b11d6133/registry/0.log"
Sep 08 21:01:54.804491 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:54.804467    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gc9sv_b691d62f-ea1a-43a9-909e-ab03a8bf9435/node-ca/0.log"
Sep 08 21:01:55.781246 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:55.781207    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7_f5be66a0-cfc9-4745-a2a6-83f195bfe307/istio-proxy/0.log"
Sep 08 21:01:56.217277 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:56.217249    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bf757d869-l5cr4_e6468cbd-3da7-4625-80d4-8c4c2ef96d89/kube-auth-proxy/0.log"
Sep 08 21:01:57.261740 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:57.261708    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ck7k5_0b838c12-7a7d-432e-ad79-d05e1104643f/serve-healthcheck-canary/0.log"
Sep 08 21:01:57.785434 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:57.785403    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-nnl6n_2d59940d-9509-4afb-92bd-0a37c249aaa5/insights-operator/0.log"
Sep 08 21:01:57.785893 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:57.785877    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-nnl6n_2d59940d-9509-4afb-92bd-0a37c249aaa5/insights-operator/1.log"
Sep 08 21:01:57.805943 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:57.805920    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hnccs_d4d58a5e-85e0-4276-a8d5-869820c7f439/kube-rbac-proxy/0.log"
Sep 08 21:01:57.826743 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:57.826718    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hnccs_d4d58a5e-85e0-4276-a8d5-869820c7f439/exporter/0.log"
Sep 08 21:01:57.846926 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:57.846894    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hnccs_d4d58a5e-85e0-4276-a8d5-869820c7f439/extractor/0.log"
Sep 08 21:01:58.757688 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:01:58.757656    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-w75tg/perf-node-gather-daemonset-qdll4"
Sep 08 21:02:00.431590 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:00.431558    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-75tzr_e235df69-f4bb-4205-8e69-bccafc3b1eec/server/0.log"
Sep 08 21:02:00.508740 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:00.508703    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-tvxbb_30c86469-f125-4b88-9219-11e5745f58c1/manager/0.log"
Sep 08 21:02:00.688645 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:00.688565    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbd9f8d48-gslsc_2b8d1349-f513-443d-91a4-fa50453cc156/manager/0.log"
Sep 08 21:02:00.709241 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:00.709212    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbd9f8d48-gslsc_2b8d1349-f513-443d-91a4-fa50453cc156/copy-manifests/0.log"
Sep 08 21:02:02.281845 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:02.281753    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57c49bb89d-snfvp_96fe2364-3ed4-4515-838f-736edf76ee93/manager/0.log"
Sep 08 21:02:02.308145 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:02.308110    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-2b7xh_7a613109-4ccd-48a1-800a-18ff438653da/openshift-lws-operator/0.log"
Sep 08 21:02:02.776764 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:02.776732    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7_f5be66a0-cfc9-4745-a2a6-83f195bfe307/istio-proxy/0.log"
Sep 08 21:02:03.194144 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:03.194111    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bf757d869-l5cr4_e6468cbd-3da7-4625-80d4-8c4c2ef96d89/kube-auth-proxy/0.log"
Sep 08 21:02:04.782303 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:04.782262    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-qxkl7_f5be66a0-cfc9-4745-a2a6-83f195bfe307/istio-proxy/0.log"
Sep 08 21:02:05.208602 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:05.208566    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bf757d869-l5cr4_e6468cbd-3da7-4625-80d4-8c4c2ef96d89/kube-auth-proxy/0.log"
Sep 08 21:02:08.978788 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:08.978750    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gnm8w_df4a9ef6-3242-48f7-b518-6a19f8a3aedd/kube-multus-additional-cni-plugins/0.log"
Sep 08 21:02:09.001887 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:09.001853    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gnm8w_df4a9ef6-3242-48f7-b518-6a19f8a3aedd/egress-router-binary-copy/0.log"
Sep 08 21:02:09.021885 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:09.021835    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gnm8w_df4a9ef6-3242-48f7-b518-6a19f8a3aedd/cni-plugins/0.log"
Sep 08 21:02:09.041551 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:09.041518    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gnm8w_df4a9ef6-3242-48f7-b518-6a19f8a3aedd/bond-cni-plugin/0.log"
Sep 08 21:02:09.062530 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:09.062485    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gnm8w_df4a9ef6-3242-48f7-b518-6a19f8a3aedd/routeoverride-cni/0.log"
Sep 08 21:02:09.082848 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:09.082822    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gnm8w_df4a9ef6-3242-48f7-b518-6a19f8a3aedd/whereabouts-cni-bincopy/0.log"
Sep 08 21:02:09.104849 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:09.104820    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gnm8w_df4a9ef6-3242-48f7-b518-6a19f8a3aedd/whereabouts-cni/0.log"
Sep 08 21:02:09.300007 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:09.299899    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-bghvs_1ddd3b60-46d5-4acf-a364-110c0486e506/kube-multus/0.log"
Sep 08 21:02:09.424749 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:09.424724    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vlx4d_642122c1-c2a9-4fe3-aade-12ac05cecf17/network-metrics-daemon/0.log"
Sep 08 21:02:09.445187 ip-10-0-136-142 kubenswrapper[2580]: I0908 21:02:09.445145    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vlx4d_642122c1-c2a9-4fe3-aade-12ac05cecf17/kube-rbac-proxy/0.log"